• [x] towards #8844

In the tests, we've been calling np.cross with vectors of 2 or 3 dimensions, numpy>=2 will deprecate 2D vectors (plus, we're now raising on warnings). Thus, we 0-pad the inputs before generating the expected result (which generally should not change the outcome of the tests).

For a later PR: add tests to check if xr.cross works if more than a single dimension is present, and pre-compute the expected result. Also, for property-based testing: the cross-product of two vectors is perpendicular to both input vectors (use the dot product to check that), and its length (l2-norm) is the product of the lengths of the input vectors.

- [x] towards #8844

• replace numpy.core.finfo with numpy.finfo
• add dtype and copy parameters to all definitions of __array__

In trying to install packages that are compatible with numpy>=2 I added several projects that are built in CI without build isolation (so that they will be built with the nightly version of numpy). That was a temporary workaround, so we should start thinking about cleaning this up.

As it seems numcodecs is now compatible (or uses less of numpy in compiled code, not sure), this is an attempt to see if CI works if we use the version from conda-forge.

bottleneck and cftime now build against numpy>=2.0.0rc1, so we can stop building them without build isolation.

Turns out pydap also needs to build with numpy>=2. Until it does, we should remove it from the upstream-dev environment. Also, numcodecs build-depends on setuptools-scm.

And finally, the h5py nightlies might support numpy>=2 (h5py>=3.11 supposedly is numpy>=2 compatible), so once again I'll try and see if CI passes.

• [x] towards #8844

FYI @aulemahal, numpy.NaN will be removed in the upcoming numpy=2.0 release.

• [x] follow-up to #8603

• [x] follow-up to #8846

pandas recently removed the deprecated unit kwarg to TimedeltaIndex.

• [x] towards #8844

There's a couple of accumulated failures now, including a crash because pandas apparently depends on pyarrow now, which on conda-forge is not built for numpy>=2.0.

• [x] closes #8748

After changing the behavior of the implementation of *_like in pint, it seems comparisons fail now. As it turns out, this is because we're using zeros_like to return all-False arrays from isnull for input with non-nullable dtypes.

I'd argue that python full_like(data, dtype=bool, fill_value=False) is a little bit easier to understand than python zeros_like(data, dtype=bool) as the latter requires knowledge about the bit representation of False, so the change is not only to get pint to work properly.

• [x] Tests added

Another one of the numpy=2.0 fixes, this time numpy.core.multiarray.normalize_axis_index has been moved to numpy.lib.array_utils (and apparently this is the first time it has been officially exposed as public API).

Since as far as I remember numpy is working on removing numpy.core entirely, we might also want to change our usage of defchararray (in the formatting tests). Not sure how, though.

• [x] Towards #8091

• [x] Closes #8580

(namely, additional CI and upstream-dev CI)

python=3.11 has been released more than a year ago and python=3.12 is out as well, which means that it is a good idea to migrate sooner than later.

Regarding python=3.12, usually it is numba that keeps us from testing on a new python version for some time, where numbagg and sparse are the only dependencies that would use it. Should we create a environment without those two dependencies and switch back to the normal one once numba supports the new python version?

We still have the special environment files for python>=3.11 because the normal ones still include cdms2. We deprecated that back in May – not sure which release that ended up in – but since cdms2 will be abandoned end of this year, that's when we're free to drop support and merge both environments (though maybe we can justify dropping support earlier?)

Not sure if this is something that we'll only see on main after the next release of pandas (if ever), though.

I also moved the hdf5 warning to xarray/tests/__init__.py, as that is usually the source of these warnings.

• [x] towards #8091

numpy recently added __array_namespace__ to the ndarray class, which returns the main numpy module. However, that does not yet provide a couple of functions, in this case we need concat. This adds a additional condition to duck_array_ops.concat that disables using __array_namespace__ for ndarray objects.

cdms2 is going to be abandoned at the end of this year, the recommended replacement is xcdat. We have deprecated our conversion functions in 2023.06.0 (release on June 23) which was less than 6 months ago, but given that cdms2 is already not installable on some architectures it makes sense to remove earlier.

This also appears to allow us to remove the special python=3.11 environment files.

• [x] Follow-up to #7876, closes #8419
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

With NEP 51 (and the changes to numpy main), scalar types no longer pretend to be standard python types in their string representation. This fixes most of the errors in the tests but there are still a few remaining in the doctests (in particular, doctests for private plotting utils).

• [x] towards #8091

The docs have been failing because of a malformatted docstring we inherit from pandas, and this caused us to miss another error in #8183. The fix is to avoid installing pandas=2.1.0, which should be the only version with the malformatted docstring, and to apply the missing changes from #8183 here.

• [x] Closes #8157, follow-up to #8183

• [x] towards #8091
• [x] closes #8133

Edit: looking at the relevant numpy issues, it appears numpy will stay a bit unstable for the next few weeks / months. Not sure how quickly we should try to adapt to those changes.

• [x] follow-up to #7415

It seems in a previous PR I "temporarily" pinned numpy to get CI to pass, but then forgot to unpin later and merged it as-is. As a result, we have not been running the main CI with numpy>=1.24 ever since, even though now numpy=1.25 has been around for a while.

Indexes contains one entry per coordinate, even if there are indexes that index multiple coordinates. This deduplicates the entries, but the format is not quite clear.

• [x] follow-up to #6795
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

The formatting options we were able to come up with: 1. separate with just newlines (see e29aeb9085bc677e16dabd4a2b94cf63d06c155e): Indexes: one CustomIndex two three PandasIndex 2. mark unique indexes with a prefix to make it look like a list (see 9b90f8bceda6f012c863927865cc638e0ff3fb88): Indexes: - one CustomIndex two - three PandasIndex 3. use unicode box components (in front of the coordinate names) (see 2cec070ebf4f1958d4ffef98d7649eda21ac09a3): Indexes: ┌ one CustomIndex │ two └ three four PandasIndex 4. use unicode box components (after the coordinate names) (see 492ab47ccce8c43d264d5c759841060d33cafe4d): Indexes: one ┐ CustomIndex two │ three ┘ four PandasIndex

For the unicode box components, we can choose between the light and heavy variants.

@benbovy and I think the unicode variants (especially variant 3) are the easiest to understand, but we would need to decide whether we care about terminals that don't support unicode.

Edit: in the meeting we decided that support for the subsection of unicode should be common enough that we can use it. I'll clean this PR up to implement option 3, then.

RTD is deprecating the feature flag we made use of before.

PyPI introduced the concept of "trusted publishers" a few months ago, which allows requesting short-lived API tokens for trusted publishing services (such as GHA, in our case).

Someone with the appropriate rights will have to enable this on PyPI, and I will do the same for TestPyPI.

Storing a reference to the function on the class tells the garbage collector to not collect the function before the class, such that any instance can safely complete its __del__.

No tests because I don't know how to properly test this. Any ideas?

• [x] Closes #7814
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

With the recent addition of the workflow to upload nightly releases to anaconda.org/scientific-python-nightly-wheels, we don't really need the TestPyPI workflow anymore, especially since PyPI instances are not designed to automatically delete very old releases.

• [x] Follow-up to #7863 and #7865
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

numba=0.57.0 has been out for quite some time already and is available on conda-forge as-of last week, which means we can almost retire the separate python=3.11 environment file.

I'm not sure what to do about cdms2 (we will see if that fails in CI), but in any case we should just deprecate and remove any functions that use it: if I understand correctly, cdms2 is in maintenance mode until the end of the year and will be discontinued afterwards.

The provision-with-micromamba action has been deprecated and will not receive any further releases. It is replaced by the setup-micromamba action, which does basically the same thing, but with a different implementation and a few changes to the configuration.

• [x] Closes #7877

As the cdms2 library has been deprecated and will be retired at the end of this year, maintaining conversion functions does not make sense anymore. Additionally, one of the tests is currently failing on the upstream-dev CI (cdms2 is incompatible with a recent change to numpy), and it seems unlikely this will ever be fixed (and there's no python=3.11 release, blocking us from merging the py311 environment with the default one).

cc @tomvothecoder for visibility

• [x] Closes #7707
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

numpy has recently changed the result of np.cos(0) by a very small value, which makes our tests break.

I'm not really sure what the best fix is, so I split the changes into two parts: the first commit uses python xr.testing.assert_allclose(a + 1, np.cos(a)) to test the result, while the second commit uses python expected = xr.full_like(a, fill_value=np.cos(0), dtype=float) actual = np.cos(a) xr.testing.assert_identical(actual, expected)

• [x] towards #7707

• [x] towards #7707

pint recently improved the support for wrapping dask, breaking our older tests. With this change, we basically require the newest version of pint (pint>=0.21.1, which should be released pretty soon) to interact with dask, as 0.21.0 did break our use of np.allclose and np.isclose. However, I guess dask support has never been properly tested and thus should be considered "experimental", anyways.

As discussed in #7765, this extends the policy months by 6 to a total of 30 months. With that, the support for a python version can be removed as soon as the next version is at least 30 months old. Together with the 12 month release cycle python has, we get the 42 month release window from NEP 29.

Note that this is still missing the release overview proposed in #7765, I'm still thinking about how to best implement the automatic update / formatting, and how to coordinate it with the (still manual) version overrides.

• [x] towards #7765, closes #7777
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Apparently, in addition to formatting notebooks, black-jupyter does exactly the same thing as black.

As a follow-up to #7724, this makes the what's new entry a bit more precise.

As mentioned in https://github.com/pydata/xarray/issues/7716#issuecomment-1497623839, this tries to unpin pandas.

Apparently, read_binary has been marked as deprecated in python=3.11, and is to be replaced by importlib.resources.files, which has been available since python=3.9. Since we dropped support for python=3.8 a while ago, we can safely follow the instructions in the deprecation warning.

The macos 3.11 CI seems stall very often at the moment, which makes it hit the 6 hours mark and get cancelled. Since our tests should never run that long (the ubuntu runners usually take between 15-20 minutes), I'm introducing a pretty generous timeout of 5 minutes. By comparison, we already have a timeout of 60 seconds in the upstream-dev CI, but that's on a ubuntu runner which usually is much faster than any of the macos / windows runners.

Tests that time out raise an error, which might help us with figuring out which test it is that stalls, and also if we can do anything about that.

In one of the recent setuptools releases pkg_resources finally got deprecated in favor of the importlib.* modules. We don't use pkg_resources directly, so the DeprecationWarning that causes the doctest CI to fail is from a dependency. As such, the only way to get it to work again is to ignore the warning until the upstream packages have released versions that don't use/import pkg_resources.

Most notably: - bump python to 3.10 - bump sphinx to at least 5.0 - remove the pydata-sphinx-theme pin: sphinx-book-theme pins to a exact minor version so pinning as well does not change anything - xref https://github.com/executablebooks/sphinx-book-theme/issues/686

~Edit: it seems this is blocked by sphinx-book-theme pinning sphinx to >=3,<5. They already changed the pin, we're just waiting on a release~

Since the last time the RTD configuration was updated, a few things have changed: - the OS image is now a bit old - we can tell git (and thus setuptools_scm) to ignore changes by RTD

If I read the documentation / history of RTD correctly, they removed the DONT_SHALLOW_CLONE feature flag from documentation (it still is enabled and works for this repository, though), so we might have to migrate to adding a post_checkout build job that does that soon.

We can't use the example grep options because we usually enclose tags with [], which grep will interpret as character classes.

RTD somewhat recently introduced the build.jobs setting, which allows skipping builds (technically it's more of a "automated cancel" with a special error code instead of a skip) on user-defined conditions. We can use that to manually "skip" builds we don't need a documentation build for.

Edit: the only downside seems to be that the build is not actually marked as "skipped" Edit2: apparently that's a bug that's being worked on, see readthedocs/readthedocs.org#9807

This builds on #7353, which found that cdms2 and numba (and thus also numbagg and sparse) don't yet support python=3.11.

In order to still test that we support python=3.11 but without dropping those dependencies from the other environments, this adds separate environment files, which should be removed once cdms2 and numba support python=3.11.

It seems there is a numbagg package on conda-forge now.

Not sure what to do about the min-all-deps CI, but given that the most recent version of numbagg happened more than 12 months ago (more than 18 months, even) maybe we can just bump it to the version on conda-forge?

• [x] opposite of #7311
• [x] closes #7306

Using the numpy HEAD together with numba is, in general, not supported (see numba/numba#8615). So in order to avoid the current failures and still be able to test numpy HEAD, this (temporarily) removes numbagg and numba (pulled in through numbagg), from the upstream-dev CI.

I ran into this while trying DataTree.pad, which silently dropped the attrs, even with keep_attrs=True.

• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

The type checking CI has started to fail with the new upgrade. In order not to disturb unrelated PRs, this pins mypy to a known good version (mypy<0.990).

Like most github actions (or at least all the official actions/* ones), issue-from-pytest-log maintains a moving release tag that points to the most recently released version.

In order to decrease the amount of updating PRs, this makes use of that tag.

While working on #7241, I realized that I forgot to specify the python in our main CI, with the effect that we've been testing python 3.10 twice for each OS since the introduction of the micromamba action.

I'm not sure if we actually need the dedicated "report" job, or whether adding an additional step to the main ci job would suffice?

I can see two reasons for a separate job: 1. we get to control the python version independently from the version of the main job 2. we upload the reportlog files as artifacts

I think if we can change the action to abort if it is run on a python version it does not support the first concern would not matter anymore, and for 2 we might just keep the "upload artifact" action (but is it even possible to manually access artifacts? If not we might not even need the upload).

Since I don't think either is a major concern, I went ahead and joined the jobs.

• [x] Closes #6810

In order to follow the minor releases of ci-trigger, we can use the new v1 release tag that will always point to the most recent release.

To see the effect, try this:

~The repr looks a bit strange because I've been borrowing the variable CSS classes.~ Edit: @benbovy fixed that for me

Also, the discussion about what _repr_inline_ should include from #7183 is relevant here as well.

• [x] Follow-up to #6795, depends on #7183
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Also, some tests for the index summarizer.

• [x] Follow-up to #6795
• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

With the flexible indexes refactor indexes have become much more important, which means we should include them in the reprs of DataArray and Dataset objects.

This is a initial attempt, covering only the string reprs, with a few unanswered questions: - how do we format indexes? Do we delegate to their __repr__ or some other method? - should we skip PandasIndex and PandasMultiIndex? - how do we present indexes that wrap multiple columns? At the moment, they are duplicated (see also the discussion in #6392) - what do we do with the index marker in the coords repr?

(also, how do we best test this?)

• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

We started to pin the version of black used in the environment of blackdoc, but the version becomes out-of-date pretty quickly. The new hook I'm adding here is still experimental, but pretty limited in what it can destroy (the pre-commit configuration) so for now we can just review any new autoupdate PRs from the pre-commit-ci a bit more thoroughly.

At the moment, nanprod and nansum will replace any nan values with 1 for nanprod or 0 for nansum. For pint, inserting dimensionless values into quantities is explicitly not allowed, so our version of nanprod cannot be used on pint quantities (0 is a exception so nansum does work).

I'm proposing to rely on numpy's version for that, which would allow pint to customize the behavior.

We could probably replace the default indexes with cudf indexes, but with pandas indexes this test doesn't make too much sense.

The reason the changes introduced in #6542 caused timeouts is that they redefined ds to a bigger dataset, which would then be used to demonstrate to_dict.

The fix is to explicitly set the dataset before calling to_dict, which also makes that section a bit easier to follow.

• [x] Closes #6720

• [x] Closes #6698

We currently use the output of pytest to generate our test results, which is both fragile and does not detect import errors on test collection.

Instead, we can use pytest-reportlog to generate a machine-readable file that is easy to parse (junit XML would probably work, too, but apparently does not contain all the different failure modes of pytest).

The new script will collect failure summaries like the old version, but it should be fairly easy to create a fancy report with more information.

I'm not sure if this is exactly equal to what we had before, but we might be able to save 3-4 minutes of CI time with this.

• [x] supersedes and closes #6544

• [x] follow-up to #4947

The only thing it contains so far is the known regression, do we want to have a summary of the most notable changes like in the previous releases?

• [x] towards #6645

cc @pydata/xarray

Hopefully the final follow-up to #6660.

As per https://github.com/pydata/xarray/issues/6659#issuecomment-1148285401, don't invoke git restore.

• [x] Closes #6659

This works around a incompatibility between setuptools>=60 and setuptools_scm.

• [x] Closes #6659

follow-up to #6660

This adds a new workflow that uploads every commit to main as a new wheel to TestPyPI. No tests, though, so those wheels might be broken (but that's fine, I guess).

Should we document this somewhere, like the install guide?

• [x] Closes #6659
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

nbconvert released a fixed version today, so we can remove the pin of jinja2.

• [x] Follow-up to #6415

(detected while trying to implement a PintMetaIndex in xarray-contrib/pint-xarray#163)

This fixes position-based indexing of duck dask arrays: ``` python

In [1]: import xarray as xr ...: impIn [1]: import xarray as xr ...: import dask.array as da ...: import pint ...: ...: ureg = pint.UnitRegistry(force_ndarray_like=True) ...: ...: a = da.zeros((20, 20), chunks=(10, 10)) ...: q = ureg.Quantity(a, "m") ...: ...: arr1 = xr.DataArray(a, dims=("x", "y")) ...: arr2 = xr.DataArray(q, dims=("x", "y"))

In [2]: arr1.isel(x=[0, 2, 4], y=[1, 3, 5]) Out[2]: <xarray.DataArray 'zeros_like-d81259c3a77e6dff3e60975e2afe4ff9' (x: 3, y: 3)> dask.array<getitem, shape=(3, 3), dtype=float64, chunksize=(3, 3), chunktype=numpy.ndarray> Dimensions without coordinates: x, y

In [3]: arr2.isel(x=[0, 2, 4], y=[1, 3, 5])

NotImplementedError Traceback (most recent call last) Input In [3], in <module> ----> 1 arr2.isel(x=[0, 2, 4], y=[1, 3, 5])

File .../xarray/core/dataarray.py:1220, in DataArray.isel(self, indexers, drop, missing_dims, **indexers_kwargs) 1215 return self._from_temp_dataset(ds) 1217 # Much faster algorithm for when all indexers are ints, slices, one-dimensional 1218 # lists, or zero or one-dimensional np.ndarray's -> 1220 variable = self._variable.isel(indexers, missing_dims=missing_dims) 1221 indexes, index_variables = isel_indexes(self.xindexes, indexers) 1223 coords = {}

File .../xarray/core/variable.py:1172, in Variable.isel(self, indexers, missing_dims, **indexers_kwargs) 1169 indexers = drop_dims_from_indexers(indexers, self.dims, missing_dims) 1171 key = tuple(indexers.get(dim, slice(None)) for dim in self.dims) -> 1172 return self[key]

File .../xarray/core/variable.py:765, in Variable.getitem(self, key) 752 """Return a new Variable object whose contents are consistent with 753 getting the provided key from the underlying data. 754 (...) 762 array x.values directly. 763 """ 764 dims, indexer, new_order = self._broadcast_indexes(key) --> 765 data = as_indexable(self._data)[indexer] 766 if new_order: 767 data = np.moveaxis(data, range(len(new_order)), new_order)

File .../xarray/core/indexing.py:1269, in NumpyIndexingAdapter.getitem(self, key) 1267 def getitem(self, key): 1268 array, key = self._indexing_array_and_key(key) -> 1269 return array[key]

File .../lib/python3.9/site-packages/pint/quantity.py:1899, in Quantity.getitem(self, key) 1897 def getitem(self, key): 1898 try: -> 1899 return type(self)(self._magnitude[key], self._units) 1900 except PintTypeError: 1901 raise

File .../lib/python3.9/site-packages/dask/array/core.py:1892, in Array.getitem(self, index) 1889 return self 1891 out = "getitem-" + tokenize(self, index2) -> 1892 dsk, chunks = slice_array(out, self.name, self.chunks, index2, self.itemsize) 1894 graph = HighLevelGraph.from_collections(out, dsk, dependencies=[self]) 1896 meta = meta_from_array(self._meta, ndim=len(chunks))

File .../lib/python3.9/site-packages/dask/array/slicing.py:174, in slice_array(out_name, in_name, blockdims, index, itemsize) 171 index += (slice(None, None, None),) * missing 173 # Pass down to next function --> 174 dsk_out, bd_out = slice_with_newaxes(out_name, in_name, blockdims, index, itemsize) 176 bd_out = tuple(map(tuple, bd_out)) 177 return dsk_out, bd_out

File .../lib/python3.9/site-packages/dask/array/slicing.py:196, in slice_with_newaxes(out_name, in_name, blockdims, index, itemsize) 193 where_none[i] -= n 195 # Pass down and do work --> 196 dsk, blockdims2 = slice_wrap_lists(out_name, in_name, blockdims, index2, itemsize) 198 if where_none: 199 expand = expander(where_none)

File .../lib/python3.9/site-packages/dask/array/slicing.py:242, in slice_wrap_lists(out_name, in_name, blockdims, index, itemsize) 238 where_list = [ 239 i for i, ind in enumerate(index) if is_arraylike(ind) and ind.ndim > 0 240 ] 241 if len(where_list) > 1: --> 242 raise NotImplementedError("Don't yet support nd fancy indexing") 243 # Is the single list an empty list? In this case just treat it as a zero 244 # length slice 245 if where_list and not index[where_list[0]].size:

NotImplementedError: Don't yet support nd fancy indexing ```

• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

sphinx is now close to releasing 4.5, but we're still pinning it to sphinx<4.

Along with it, this updates our RTD configuration.

Since the dependency issue has been fixed upstream, this reverts pydata/xarray#6350

This can be reverted once the nbconvert package on conda-forge has been updated.

Since that question came up at least twice, this allows changing the behavior of where using the keep_attrs option. Dataset.where and DataArray.where always use keep_attrs=True (set in ops.where_method), so we could probably copy that.

Edit: we should also document that only the attributes of cond will be preserved.

• [x] Closes #4682, closes #4141
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

In #5845, we accidentally broke the detection of the backends. Since this has a big impact we probably need to release v0.20.1 very soon.

I'm not sure if it's possible to add tests for this, though.

• [x] Closes #5930
• [ ] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Reverts pydata/xarray#5532

Not being able to pass entrypoints to the engine parameter is a bug which should be fixed by #5684.

• [x] closes #5777

add cupy, sparse and fsspec to the output of xr.show_versions()

• [x] Passes pre-commit run --all-files

As this came up in #5424 (and because I can't seem to reliably reproduce expected values in #4972 if it is enabled) this adds a option to disable bottleneck, even if it is installed.

In #5424 it was suggested to also allow replacing bottleneck with numbagg. If that's something we want (and numbagg supports the operations we need) I can try looking into renaming the new option to something like xr.set_options(accelerate_with="bottleneck") (or something else, if someone has a great idea).

Tests are missing because I have no idea how to check that this works (except by mocking bottleneck).

• [x] Closes #5424
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• [x] Passes pre-commit run --all-files

As far as I can tell, the examples on binder currently fail because pooch is not installed.

• [x] Passes pre-commit run --all-files

As discussed in the meeting yesterday, here's the PR for the scheduled deprecations. We also have quite a few deprecations which are not scheduled... we should check which of them can be removed / completed, but that can wait until after the release.

I tried to keep the changes for each individual deprecation in their own commit, which should make rolling back easier in case we decide removing / completing a specific deprecation is too aggressive.

whats-new.rst still needs to be updated, but I'll leave that until after the code changes have been approved.

• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

As discussed this pins fsspec to known good versions, which should hopefully make sure the CI passes.

cc @martindurant

follow-up to #5468. We removed the external version file but the sync script is still run in CI, which obviously fails and causes the report to be stripped from the PR comment.

For a future PR / investigation: pre-commit allows installing dependencies from the language's package index (e.g. PyPI for python), but packages installed using that method are not updated until the hook is updated / reinstalled. For blackdoc we were thinking about pinning black at some point so we could work around that by synchronizing the version with the hook version, but this is probably more work than it is worth. I guess this means we have to wait on pre-commit to fix this and in the meantime we'll probably have to tell people to clean their pre-commit caches on updates for black and numpy...

towards #5516. With this, all references to the master branch have been updated to point to main.

The stable builds on RTD are currently failing. latest is failing, too, but that might be fixed by #5476 (I can also cherry-pick the merge commit to main, though).

Apparently the test coverage was not high enough: we were passing a Variable object to numpy.gradient. This only adds tests for DataArray.differentiate because Dataset.differentiate doesn't seem to be as easy to check (the axis might be different).

• [x] Closes xarray-contrib/pint-xarray#106
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Allowing to pass a function to combine_attrs is important if a user wants to do something the builtin merge strategies don't support.

• [x] towards #3891
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

In #5235 we seem to have accidentally broken the repr of Variable objects with display_expand_data=False. I don't think Variable objects benefit from the collapsed view, so the easiest fix seems to be to always use short_data_repr for Variable objects.

• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

In preparation of pushing these upstream, this increases the priority of the _repr_inline_ implementation of dask or sparse (once they are implemented) over our own custom functions.

• [ ] Tests added
• [x] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

follow-up to #5343, which got the upstream-dev CI to run but unfortunately not the "report" job.

• [x] Passes pre-commit run --all-files

5267 seems to accidentally have disabled the scheduled CI.

• [x] Passes pre-commit run --all-files

Since it seems difficult to fix the situation using the package manager, this improves the error message by explicitly suggesting to install pooch.

• [x] partial fix for #5291
• [ ] Tests added
• [x] Passes pre-commit run --all-files

This implements the easier fix described in #5241 which doesn't involve creating a personal access token (which I think is tied to a specific account, even though it is set on a repository). Of course, it would be great to have the Additional CI and linting workflows run on PRs created by the action.

• [x] closes #5241
• [x] Passes pre-commit run --all-files

Follow-up to #5239.

• [x] Passes pre-commit run --all-files

follow-up to #5304

• [x] Passes pre-commit run --all-files

Follow-up to #5303. The feedstock has been updated (thanks for the ping, @kmuehlbauer), so we can unpin it again.

• [x] Passes pre-commit run --all-files

This can be reverted once the nbsphinx feedstock copies the library's pins.

• [x] Closes #5299
• [x] Passes pre-commit run --all-files

As discussed this extends keep_attrs in apply_ufunc to also accept merge strategy names and use merge_attrs (making it easier to add more merge strategies).

Handling attrs on variables / coords is still missing.

• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst