• [x] Closes #7377
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Just saw your message in the related issue @max-sixty. This is what I came up with earlier. I also did a quick test, comparing the calculation with and without using numbagg for a dummy 3D DataArray. I was only wondering if the default usage of numbagg (given that it's available and method='linear') should be noted somewhere in the docstrings and/or the docs in general.

What happened?

Hi all, as the title already summarizes, I'm running into performance issues when aggregating over the time-dimension of a 3D DataArray using the quantiles method with skipna=True. See the section below for some dummy data that represents what I'm working with (e.g., similar to this). Aggregating over the time-dimension of this dummy data I'm getting the following wall times:

| | | | | --------------- | --------------- | --------------- | | 1 | da.median(dim='time', skipna=True) | 1.35 s | | 2 | da.quantile(0.95, dim='time', skipna=False) | 5.95 s | | 3 | da.quantile(0.95, dim='time', skipna=True) | 6 min 6s |

I'm currently using a compute node with 40 CPUs and 180 GB RAM. Here is what the resource utilization looks like. First small bump are 1 and 2. Second longer peak is 3.

In this small example, the process at least finishes after a few seconds. With my actual dataset the quantile calculation takes hours...

I guess the following issue is relevant and should be revived: https://github.com/numpy/numpy/issues/16575

Are there any possible work-arounds?

What did you expect to happen?

No response

Minimal Complete Verifiable Example

```Python import pandas as pd import numpy as np import xarray as xr

Create dummy data with 20% random NaNs

size_spatial = 2000 size_temporal = 20 n_nan = int(size_spatial*20.2)

time = pd.date_range("2000-01-01", periods=size_temporal) lat = np.random.uniform(low=-90, high=90, size=size_spatial) lon = np.random.uniform(low=-180, high=180, size=size_spatial) data = np.random.rand(size_temporal, size_spatial, size_spatial) index_nan = np.random.choice(data.size, n_nan, replace=False) data.ravel()[index_nan] = np.nan

Create DataArray

da = xr.DataArray(data=data, dims=['time', 'x', 'y'], coords={'time': time, 'x': lon, 'y': lat}, attrs={'nodata': np.nan})

Calculate 95th quantile over time-dimension

da.quantile(0.95, dim='time', skipna=True) ```

MVCE confirmation

• [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [x] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

The URL in the section about creating a local development environment was pointing to itself. The new URL is pointing to the (I assume) correct section further down in the same guide.

What happened?

When performing groupby-/resample-reduce operations (e.g., ds.resample(time="6h").mean() as shown here) the alignment of chunks is not preserved when flox is disabled:

...whereas the alignment is preserved when flox is enabled:

What did you expect to happen?

The alignment of chunks is preserved whether using flox or not.

Minimal Complete Verifiable Example

```Python import pandas as pd import numpy as np import xarray as xr

size_spatial = 1000 size_temporal = 200 time = pd.date_range("2000-01-01", periods=size_temporal, freq='h') lat = np.random.uniform(low=-90, high=90, size=size_spatial) lon = np.random.uniform(low=-180, high=180, size=size_spatial) data = np.random.rand(size_temporal, size_spatial, size_spatial)

da = xr.DataArray(data=data, dims=['time', 'x', 'y'], coords={'time': time, 'x': lon, 'y': lat}).chunk({'time': -1, 'x': 'auto', 'y': 'auto'})

Chunk alignment not preserved

with xr.set_options(use_flox=False): da_1 = da.copy(deep=True) da_1 = da_1.resample(time="6h").mean()

Chunk alignment preserved

with xr.set_options(use_flox=True): da_2 = da.copy(deep=True) da_2 = da_2.resample(time="6h").mean() ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

No response

Anything else we need to know?

No response

Environment