• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Small follow up to #8877. If we're going to drop the indices anyways for region writes, we may as well not raise if they are still in the dataset. This makes the user experience of region writes simpler:

```python ds = xr.tutorial.open_dataset("air_temperature") ds.to_zarr("test.zarr") region = {"time": slice(0, 10)}

This fails unless we remember to ds.drop_vars(["lat", "lon"])

ds.isel(**region).to_zarr("test.zarr", region=region) ```

I find this annoying because I often have a dataset with a bunch of unrelated indexes and have to remember which ones to drop, or use some verbose set logic. I thought #8877 might have already done this, but not quite. By just reordering the point at which we drop indices, we can now skip this. We still raise if data vars are passed that don't overlap with the region.

cc @dcherian

• [x] Closes https://github.com/xarray-contrib/xeofs/issues/148
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

The change from https://github.com/pydata/xarray/pull/8672 mostly fixed the issue of serializing a reset multiindex in the backends, but there was an additional niche issue that turned up in xeofs that was causing serialization to still fail on the zarr backend.

The issue is that zarr is the only backend that uses a custom version of encode_cf_variable called encode_zarr_variable, and the way this gets called we don't pass through the name of the variable before running ensure_not_multiindex.

As a minimal fix, this PR just passes name through as an additional arg to the general encode_variable function. See @benbovy's comment that maybe we should actually unwrap the level coordinate in reset_index and clean up the checks in ensure_not_multiindex, but I wasn't able to get that working easily.

The exact workflow this turned up in involves DataTree and looks like this: ```python import numpy as np import xarray as xr from datatree import DataTree

ND DataArray that gets stacked along a multiindex

da = xr.DataArray(np.ones((3, 3)), coords={"dim1": [1, 2, 3], "dim2": [4, 5, 6]}) da = da.stack(feature=["dim1", "dim2"])

Extract just the stacked coordinates for saving in a dataset

ds = xr.Dataset(data_vars={"feature": da.feature})

Reset the multiindex, which should make things serializable

ds = ds.reset_index("feature") dt1 = DataTree() dt2 = DataTree(name="feature", data=ds) dt1["foo"] = dt2

Somehow in this step, dt1.foo.feature.dim1.variable becomes an IndexVariable again

print(type(dt1.foo.feature.dim1.variable))

Works

dt1.to_netcdf("test.nc", mode="w")

Fails

dt1.to_zarr("test.zarr", mode="w") ```

But we can reproduce in xarray with the test added here.

• [x] Closes #7702, #8421
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

A quick pass at implementing these two improvements for zarr region writes:

1. allow passing region={dim: "auto"}, which opens the existing zarr store and identifies the correct slice to write to, using a variation of the approach suggested by @DahnJ here. We also check for non-matching coordinates and non-contiguous indices.
2. automatically transpose dimensions if they otherwise match the existing store but are out of order

• [x] Closes #7362
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• [x] Closes #7220
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Reverts the getattr method used in xr.where(..., keep_attrs=True) from #6461, but keeps handling for scalar inputs. Adds some test cases to ensure consistent attribute handling.

• [x] Closes #6444
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Fixes a bug introduced by #4687 where passing a non-xarray object to x in xr.where(cond, x, y, keep_attrs=True) caused a failure. The keep_attrs callable passed to merge_attrs() tries to access the attributes of x which do not exist in this case. This fix just checks to make sure x has attributes, and if not will pass through keep_attrs=True.

• [x] Closes #6891
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• [x] Closes #4554
• [x] Closes #5629
• [x] Closes #5644
• [ ] Tests added
• [x] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Reimplement polyfit using apply_ufunc rather than dask.array.linalg.lstsq. This should solve a number of issues with memory usage and chunking that were reported on the current version of polyfit. The main downside is that variables chunked along the fitting dimension cannot be handled with this approach.

There is a bunch of fiddly code here for handling the differing outputs from np.polyfit depending on the values of the full and cov args. Depending on the performance implications, we could simplify some by keeping these in apply_ufunc and dropping later. Much of this parsing would still be required though, because the only way to get the covariances is to set cov=True, full=False.

A few minor departures from the previous implementation: 1. The rank and singular_values diagnostic variables returned by np.polyfit are now returned on a pointwise basis, since these can change depending on skipped nans. np.polyfit also returns the rcond used for each fit which I've included here. 2. As mentioned above, this breaks fitting done along a chunked dimension. To avoid regression, we could set allow_rechunk=True and warn about memory implications. 3. Changed default skipna=True, since the previous behavior seemed to be a limitation of the computational method. 4. For consistency with the previous version, I included a transpose operation to put degree as the first dimension. This is arbitrary though, and actually the opposite of how curvefit returns ordering. So we could match up with curvefit but it would be breaking for polyfit.

No new tests have been added since the previous suite was fairly comprehensive. Would be great to get some performance reports on real-world data such as the climate model detrending application in #5629.

• [x] Closes #7062
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

This just tests to make sure we at least get the same dtype whether we have a numpy or dask array.

• [x] Closes #6484
• [x] Tests added

I've often been tripped up by the stack trace noted in #6484. This PR changes two things:

1. Handles the zarr GroupNotFoundError error with a more informative FileNotFoundError, displaying the path where we didn't find a zarr store.
2. Moves the consolidated metadata warning to after the step of successfully opening the zarr with non-consolidated metadata. This way the warning isn't shown if we are actually trying to open a non-existent zarr store, in which case we only get the error above and no warning.

• [x] Closes #4300
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

This is a simple implementation of a more general curve-fitting API as discussed in #4300, using the existing scipy curve_fit functionality wrapped with apply_ufunc. It works for arbitrary user-supplied 1D functions that ingest numpy arrays. Formatting and nomenclature of the outputs was largely copied from .polyfit, but could probably be improved.