issues

Turns out pydap also needs to build with numpy>=2. Until it does, we should remove it from the upstream-dev environment. Also, numcodecs build-depends on setuptools-scm.

And finally, the h5py nightlies might support numpy>=2 (h5py>=3.11 supposedly is numpy>=2 compatible), so once again I'll try and see if CI passes.

• [x] towards #8844

• [x] Closes #8580

With NEP 51 (and the changes to numpy main), scalar types no longer pretend to be standard python types in their string representation. This fixes most of the errors in the tests but there are still a few remaining in the doctests (in particular, doctests for private plotting utils).

• [x] towards #8091

The docs have been failing because of a malformatted docstring we inherit from pandas, and this caused us to miss another error in #8183. The fix is to avoid installing pandas=2.1.0, which should be the only version with the malformatted docstring, and to apply the missing changes from #8183 here.

• [x] Closes #8157, follow-up to #8183

With the flexible indexes refactor indexes have become much more important, which means we should include them in the reprs of DataArray and Dataset objects.

This is a initial attempt, covering only the string reprs, with a few unanswered questions: - how do we format indexes? Do we delegate to their __repr__ or some other method? - should we skip PandasIndex and PandasMultiIndex? - how do we present indexes that wrap multiple columns? At the moment, they are duplicated (see also the discussion in #6392) - what do we do with the index marker in the coords repr?

(also, how do we best test this?)

• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

As this came up in #5424 (and because I can't seem to reliably reproduce expected values in #4972 if it is enabled) this adds a option to disable bottleneck, even if it is installed.

In #5424 it was suggested to also allow replacing bottleneck with numbagg. If that's something we want (and numbagg supports the operations we need) I can try looking into renaming the new option to something like xr.set_options(accelerate_with="bottleneck") (or something else, if someone has a great idea).

Tests are missing because I have no idea how to check that this works (except by mocking bottleneck).

• [x] Closes #5424
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

In preparation of pushing these upstream, this increases the priority of the _repr_inline_ implementation of dask or sparse (once they are implemented) over our own custom functions.

• [ ] Tests added
• [x] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Follow-up to #4827. The current behavior is to always combine with combine_attrs="override", even if the caller requested something else.

• [x] follow-up to #4827, closes #4017
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

black changed their style (format docstrings and trailing comma handling mostly) in their new release. This simply runs that new version of black on the repository.

This is a separate PR so reviewing becomes easier.

Since it is an automatic change, this should be ready to be merged once the CI is green.

Now that the tests are cleaned up, it is time to fix the numerous "unit stripped" warnings. We'll get there by promoting all UnitStrippedWarnings to errors (using filterwarnings) and fixing / xfailing the failing tests.

• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

We currently use property to attach our built-in accessors (plot, dt and str) to the DataArray and Dataset classes. However, because property returns itself when trying to access it as a class attribute, this does not work when generating the documentation (sphinx inspects classes, not class instances).

This adds a property-like descriptor that works on both classes and instance objects (the name could be more descriptive) and uses that to document the Dataset.plot.* and DataArray.plot.* methods (see the rendered documentation). I have not been able to get sphinx to work with _PlotMethods.__slots__, though.

A few questions / comments on this: 1. I noticed we have DataArray.plot.__call__ and xarray.plot.plot but not DataArray.plot.plot. Is that something that is worth adding?

1. The functions decorated with the custom property define docstrings which currently are lost. Should we patch them on their return values?

2. Right now, the error message when accidentally trying to call e.g. xr.DataArray.plot.line() is not very helpful: pytb AttributeError: 'NoneType' object has no attribute 'dims'

3. Now that we can document the accessors, we need to think about how to structure api.rst. For plot and str, we could just list methods / attributes in subsections of DataArray / Dataset (or keep the Plotting / str / ... sections and add class subsections?), but dt is more complicated since it dispatches for datetime and timedelta

4. [x] Closes #3625

5. [x] Passes isort -rc . && black . && mypy . && flake8
6. [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Edit: err, it seems I used the branch I pushed to the main repository for a documentation preview for this...

The reason for those fragments is that napoleon and numpydoc somehow conflict with each other. From a quick glance I couldn't find anything that breaks without numpydoc, but I might be missing something.

For reference, here's the new API page. - [x] Closes #3178 - [x] Passes isort -rc . && black . && mypy . && flake8

This PR tries to get the pint integration tests (see #3594) to pass. To make sure this does not become a giant PR, it is limited to the top-level functions and the work on the tests for DataArray and Dataset will be done in other PRs.

I added pint master to the upstream-dev CI since that way CI finally helps finding bugs. Should I have edited py36-min-nep18 instead?

• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Also, I finally ran into a situation where I have to use assert_allclose instead of assert_equal_with_units. However, assert_allclose does not check for identical units, I think? Would it make sense to copy the code of assert_allclose and add the strict unit checking or would it be better to just use assert_allclose and something like assert extract_units(expected) == extract_units(actual)?

Failing tests list from #3594: - align: ~needs more investigation~ ~partially blocked by where, IndexVariable and a bug in assert_equal_with_units and / or the tests, also a behaviour decision on align~ works (but units in IndexVariable still fail) - combine_by_coords: ~needs more investigation~ ~partially blocked by IndexVariable and a test bug (due to #3483)~ works (but units in IndexVariable still fail) - combine_nested: ~needs more investigation~ ~blocked by reindex~ works (but units in IndexVariable still fail) - concat: ~needs more investigation~ works (but units in IndexVariable still fail) - merge: ~needs more investigation~ works (but units in IndexVariable still fail) - full_like: ~pint currently does not implement copyto~ blocked by hgrecco/pint#882 - where: ~needs more investigation~ ~unit is stripped because pint does not implement astype and test bug in test_where_dataset~ works - dot: ~pint currently does not implement einsum~ works

As a follow-up to #3516, this silences the last warnings without -n, so once the doctest issue below is fixed we could replace -n with -W --keep-going and have the CI fail should a PR introduce new warnings.

There is, however, a problem with the doctest output for the 3D result of GroupBy.quantile: since the repr of a DataArray with at least 3 dimensions contains blank lines, it complains about a block quote ending without a blank line.

Also, the complaints from #3516 about duplicate definitions of attributes were caused by the Attributes section in the docstring of DataArray. I fixed this by removing the whole section, but we probably want to first make sure this does not remove any information (or maybe someone has a different idea?).

• [x] Passes black . && mypy . && flake8

This adds the tests that were missed by #3238 and #3447.

• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This should be ready to merge, but I'm almost certain that while going through #3238 and #3447 and trying to fix obvious mistakes I will find more missing tests.

Missing tests: * [x] Dataset.broadcast_like and DataArray.broadcast_like * [x] DataArray.head, DataArray.tail and DataArray.thin

While going through the docs, I noticed several issues: * in reshaping.rst, a code block gets hidden because of a missing newline * in api.rst, the section What parts of xarray are considered public API? is referenced (I think), but the link is broken. * the code samples in DataArray.plot are not recognized as such because of missing newlines * the documentation of Dataset.plot.scatter in the API reference is broken

This PR tries to fix all of those, but while the first three issues are easy to fix, I'm not sure whether my proposed fix for the one involving scatter is correct: I've added plot.dataset_plot.scatter to plot.__all__ (exposing xr.plot.scatter) and changed api.rst to reference plot.scatter instead of Dataset.plot.scatter. It seems harmless to me, but the information that scatter only exists for Dataset is less visible. Is that the way to fix this?

• [x] Passes black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API