• [x] Solves part of #8871 by pulling out part of https://github.com/pydata/xarray/pull/8872#issuecomment-2027571714
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] ~~New functions/methods are listed in api.rst~~

TODO: - [x] Add new kwarg to DataArray.expand_dims - [ ] Add examples to docstrings? - [x] Check it actually solves the problem in #8872

What happened?

Passing no data to the Dataset constructor can result in a very unhelpful "tuple index out of range" error when this is a clear case of malformed input that we should be able to catch.

What did you expect to happen?

An error more like "tuple must be of form (dims, data[, attrs])"

Minimal Complete Verifiable Example

Python xr.Dataset({"t": ()})

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python

IndexError Traceback (most recent call last) Cell In[2], line 1 ----> 1 xr.Dataset({"t": ()})

File ~/Documents/Work/Code/xarray/xarray/core/dataset.py:693, in Dataset.init(self, data_vars, coords, attrs) 690 if isinstance(coords, Dataset): 691 coords = coords._variables --> 693 variables, coord_names, dims, indexes, _ = merge_data_and_coords( 694 data_vars, coords 695 ) 697 self._attrs = dict(attrs) if attrs else None 698 self._close = None

File ~/Documents/Work/Code/xarray/xarray/core/dataset.py:422, in merge_data_and_coords(data_vars, coords) 418 coords = create_coords_with_default_indexes(coords, data_vars) 420 # exclude coords from alignment (all variables in a Coordinates object should 421 # already be aligned together) and use coordinates' indexes to align data_vars --> 422 return merge_core( 423 [data_vars, coords], 424 compat="broadcast_equals", 425 join="outer", 426 explicit_coords=tuple(coords), 427 indexes=coords.xindexes, 428 priority_arg=1, 429 skip_align_args=[1], 430 )

File ~/Documents/Work/Code/xarray/xarray/core/merge.py:718, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value, skip_align_args) 715 for pos, obj in skip_align_objs: 716 aligned.insert(pos, obj) --> 718 collected = collect_variables_and_indexes(aligned, indexes=indexes) 719 prioritized = _get_priority_vars_and_indexes(aligned, priority_arg, compat=compat) 720 variables, out_indexes = merge_collected( 721 collected, prioritized, compat=compat, combine_attrs=combine_attrs 722 )

File ~/Documents/Work/Code/xarray/xarray/core/merge.py:358, in collect_variables_and_indexes(list_of_mappings, indexes) 355 indexes_.pop(name, None) 356 append_all(coords_, indexes_) --> 358 variable = as_variable(variable, name=name, auto_convert=False) 359 if name in indexes: 360 append(name, variable, indexes[name])

File ~/Documents/Work/Code/xarray/xarray/core/variable.py:126, in as_variable(obj, name, auto_convert) 124 obj = obj.copy(deep=False) 125 elif isinstance(obj, tuple): --> 126 if isinstance(obj[1], DataArray): 127 raise TypeError( 128 f"Variable {name!r}: Using a DataArray object to construct a variable is" 129 " ambiguous, please extract the data using the .data property." 130 ) 131 try:

IndexError: tuple index out of range ```

Anything else we need to know?

No response

Environment

Xarray main

Supercedes #8884 as a way to close #8883, in light of me having learnt that this is now allowed! https://github.com/pydata/xarray/issues/8883#issuecomment-2024645815. So this is really a follow-up to #7989.

• [x] Closes #8883
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] ~~New functions/methods are listed in api.rst~~

What happened?

It is currently possible to create a Coordinates object where a variable shares a name with a dimension, but the variable is not 1D. This is explicitly forbidden by the xarray data model.

What did you expect to happen?

If you try to pass the resulting object into the Dataset constructor you get the expected error telling you that this is forbidden, but that error should have been raised by Coordinates.__init__.

Minimal Complete Verifiable Example

```Python In [1]: from xarray.core.coordinates import Coordinates

In [2]: from xarray.core.variable import Variable

In [4]: import numpy as np

In [5]: var = Variable(data=np.arange(6).reshape(2, 3), dims=['x', 'y'])

In [6]: var Out[6]: <xarray.Variable (x: 2, y: 3)> Size: 48B array([[0, 1, 2], [3, 4, 5]])

In [7]: coords = Coordinates(coords={'x': var}, indexes={})

In [8]: coords Out[8]: Coordinates: x (x, y) int64 48B 0 1 2 3 4 5

In [10]: import xarray as xr

In [11]: ds = xr.Dataset(coords=coords)

MergeError Traceback (most recent call last) Cell In[11], line 1 ----> 1 ds = xr.Dataset(coords=coords)

File ~/Documents/Work/Code/xarray/xarray/core/dataset.py:693, in Dataset.init(self, data_vars, coords, attrs) 690 if isinstance(coords, Dataset): 691 coords = coords._variables --> 693 variables, coord_names, dims, indexes, _ = merge_data_and_coords( 694 data_vars, coords 695 ) 697 self._attrs = dict(attrs) if attrs else None 698 self._close = None

File ~/Documents/Work/Code/xarray/xarray/core/dataset.py:422, in merge_data_and_coords(data_vars, coords) 418 coords = create_coords_with_default_indexes(coords, data_vars) 420 # exclude coords from alignment (all variables in a Coordinates object should 421 # already be aligned together) and use coordinates' indexes to align data_vars --> 422 return merge_core( 423 [data_vars, coords], 424 compat="broadcast_equals", 425 join="outer", 426 explicit_coords=tuple(coords), 427 indexes=coords.xindexes, 428 priority_arg=1, 429 skip_align_args=[1], 430 )

File ~/Documents/Work/Code/xarray/xarray/core/merge.py:731, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value, skip_align_args) 729 coord_names.intersection_update(variables) 730 if explicit_coords is not None: --> 731 assert_valid_explicit_coords(variables, dims, explicit_coords) 732 coord_names.update(explicit_coords) 733 for dim, size in dims.items():

File ~/Documents/Work/Code/xarray/xarray/core/merge.py:577, in assert_valid_explicit_coords(variables, dims, explicit_coords) 575 for coord_name in explicit_coords: 576 if coord_name in dims and variables[coord_name].dims != (coord_name,): --> 577 raise MergeError( 578 f"coordinate {coord_name} shares a name with a dataset dimension, but is " 579 "not a 1D variable along that dimension. This is disallowed " 580 "by the xarray data model." 581 )

MergeError: coordinate x shares a name with a dataset dimension, but is not a 1D variable along that dimension. This is disallowed by the xarray data model. ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

No response

Anything else we need to know?

I noticed this whilst working on #8872

Environment

main

• [x] Closes #8883
• [x] Tests added
• [ ] ~~User visible changes (including notable bug fixes) are documented in whats-new.rst~~
• [ ] ~~New functions/methods are listed in api.rst~~

• [x] Hopefully closes #8619

• [x] Needed for https://github.com/tomwhite/cubed/issues/277#issuecomment-1648567431 - should have been added in #7019
• [ ] ~~Tests added~~ (would go in cubed-xarray)
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst (new ABC method will be documented on chunked array types page automatically)

Adds a new .chunksizes property to Dataset, DataArray and Variable, which returns a mapping from dimensions names to chunk sizes in all cases.

Supercedes #5846 because this PR is backwards-compatible.

• [x] Closes #5843
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Another attempt like #5961

@shoyer

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

While it's common for duck arrays to have a .astype method, this doesn't exist in the new array API standard. We now have duck_array_ops.astype to deal with this, but for some reason changing it in just a couple more places broke practically every pint test in test_units.py :confused: @keewis

Builds on top of #7019 with just one extra commit to separate out this issue.

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

GH action to automatically label new PRs according to which files they touch.

Idea stolen from dask, see https://github.com/dask/dask/pull/7506 . Their PR labelling by file/module is specified here.

(My first use of this bot so might well be a mistake.)

@max-sixty you will probably enjoy this extra automation :robot:

• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

@pydata/xarray the release notes for your approval

5889

Noticed this hadn't been completed in https://github.com/pydata/xarray/discussions/5659

• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

As of python 3.9, python dictionaries now support being merged via python c = a | b and updated via python c = a |= b see PEP 584.

xarray.Dataset is dict-like, so it would make sense to support the same syntax for merging. The way to achieve that is by adding new dunder methods to xarray.Dataset, something like

```python def or(self, other): if not isinstance(other, xr.Dataset): return NotImplemented new = xr.merge(self, other) return new

def ror(self, other): if not isinstance(other, xr.Dataset): return NotImplemented new = xr.merge(self, other) return new

def ior(self, other): self.merge(other) return self ```

The distinction between the intent of these different operators is whether a new object is returned or the original object is updated.

This would allow things like (ds1 | ds2).to_netcdf()

(This feature doesn't require python 3.9, it merely echoes a feature that is only available in 3.9+)

I noticed that the apply_ufunc tutorial notebook says that xarray.apply_ufunc uses dask.array.blockwise, but that's no longer true as of PR #4060 .

• [x] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Inspired by a question asked in #4696, but does not close that issue - [x] Passes pre-commit run --all-files - [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Does this need a test?

• [x] Closes #5073
• [ ] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

When I try to run xarray's test suite locally with pytest I've suddenly started getting this weird error:

``` (xarray-dev) tegn500@fusion192:~/Documents/Work/Code/xarray$ pytest xarray/tests/test_backends.py ==================================================================================== test session starts ===================================================================================== platform linux -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 rootdir: /home/tegn500/Documents/Work/Code/xarray, configfile: setup.cfg collected 0 items / 1 error

=========================================================================================== ERRORS =========================================================================================== __________ ERROR collecting xarray/tests/test_backends.py __________ ../../../../anaconda3/envs/xarray-dev/lib/python3.9/importlib/init.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) <frozen importlib._bootstrap>:1030: in _gcd_import ??? <frozen importlib._bootstrap>:1007: in _find_and_load ??? <frozen importlib._bootstrap>:972: in _find_and_load_unlocked ??? <frozen importlib._bootstrap>:228: in _call_with_frames_removed ??? <frozen importlib._bootstrap>:1030: in _gcd_import ??? <frozen importlib._bootstrap>:1007: in _find_and_load ??? <frozen importlib._bootstrap>:986: in _find_and_load_unlocked ??? <frozen importlib._bootstrap>:680: in _load_unlocked ??? <frozen importlib._bootstrap_external>:790: in exec_module ??? <frozen importlib._bootstrap>:228: in _call_with_frames_removed ??? xarray/tests/init.py:84: in <module> has_pint_0_15, requires_pint_0_15 = _importorskip("pint", minversion="0.15") xarray/tests/init.py:46: in _importorskip if LooseVersion(mod.version) < LooseVersion(minversion): E AttributeError: module 'pint' has no attribute 'version' ================================================================================== short test summary info =================================================================================== ERROR xarray/tests/test_backends.py - AttributeError: module 'pint' has no attribute 'version' !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ====================================================================================== 1 error in 0.88s ====================================================================================== ```

I'm not sure whether this is my fault or a problem with xarray somehow. @keewis have you seen this happen before? This is with a fresh conda environment, running locally on my laptop, and on python 3.9.2. Pint isn't even in this environment. I can force it to proceed with the tests by also catching the attribute error, i.e.

python def _importorskip(modname, minversion=None): try: mod = importlib.import_module(modname) has = True if minversion is not None: if LooseVersion(mod.__version__) < LooseVersion(minversion): raise ImportError("Minimum version not satisfied") except (ImportError, AttributeError): has = False

but I obviously shouldn't need to do that. Any ideas?

Environment:

Conda Environment:

• [x] Closes #3992
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I am interested in adding the ability to plot surface plots of 2D xarray data using matplotlib's 3D plotting function plot_surface().

This would be nice because a surface in 3D is much more useful for showing certain features of 2D data then color plots are. For example an outlier would appear as an obvious spike rather than just a single bright point as it would when using plot.imshow(). I'm not suggesting adding full 3D plotting capability, just the ability to visualise 2D data as a surface in 3D.

The code would end up allowing you to just call xr.Dataarray.plot.surface() to create something like this example from here (code here):

Obviously xarray would be used to automatically set the axes labels and title and so on.

As far as I can tell it wouldn't be too difficult to do, it would just be implemented as another 2D plotting method the same way as the Dataarray.plot.imshow(), Dataarray.plot.contour() etc methods currently are. It would require the imports python import matplotlib.pyplot as plt from mpl_toolkits.mplot3d import Axes3D but these would only need to be imported if this type of plot was chosen.

I would be interested in trying to add this myself, but I've never contributed to an open-source project before. Is this a reasonable thing for me to try? Can anyone see any immediate difficulties with this? Would I just need to have a go and then submit a pull request?

dataarray.interpolate_na was dropping attrs because even though the internal apply_ufunc call was being passed keep_attrs=True, the order of arguments index, da to apply_ufunc meant that it was trying to keep only the non-existent attrs from the first argument. I just swapped them round, and added a globally-aware keep_attrs kwarg.

• [x] Closes #3968
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Names listed in order of date added.

• [x] Closes #3892
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Finishes #2616 by renaming combine_manual -> combine_nested, combine_auto -> combine_by_coords, and renaming the arguments to open_mfdataset: combine='manual' -> combine='nested' combine='auto' -> combine='by_coords'

[x] code changes [x] docs changes [x] what's new updated

While writing some unit tests to check I had wrapped np.gradient correctly with xr.apply_ufunc, I came unstuck because my results were equivalent except for transposed dimensions. It took me a while to realise that xarray.testing.asset_equal considers two DataArrays equal only if their dimensions are in the same order, because intuitively that shouldn't matter in the context of xarray's data model.

A simple example to demonstrate what I mean: ```python

Create two functionally-equivalent dataarrays

data = np.random.randn(4, 3) da1 = xr.DataArray(data, dims=('x', 'y')) da2 = xr.DataArray(data.T, dims=('y', 'x'))

This test will fail

xarray.tests.assert_equal(da1, da2) This test fails, with output E AssertionError: <xarray.DataArray (x: 4, y: 3)> E array([[ 0.761038, 0.121675, 0.443863], E [ 0.333674, 1.494079, -0.205158], E [ 0.313068, -0.854096, -2.55299 ], E [ 0.653619, 0.864436, -0.742165]]) E Coordinates: E * x (x) int64 5 7 9 11 E * y (y) int64 1 4 6 E <xarray.DataArray (y: 3, x: 4)> E array([[ 0.761038, 0.333674, 0.313068, 0.653619], E [ 0.121675, 1.494079, -0.854096, 0.864436], E [ 0.443863, -0.205158, -2.55299 , -0.742165]]) E Coordinates: E * x (x) int64 5 7 9 11 E * y (y) int64 1 4 6 ``` even though these two DataArrays are functionally-equivalent for all xarray operations you could perform with them.

It would make certain types of unit tests simpler and clearer to have a function like python xarray.tests.assert_equivalent(da1, da2) which would return true if one DataArray can be formed from the other by transposition.

I would have thought that a test that does this would just transpose one into the shape of the other before comparison?