What happened?

To parallelize a computation over a sparse array, I want to create chunks after stacking the array and dropping fill values. Such an array has a multiindex, which breaks the map_blocks method (with an unhelpful error). The AssertionError raised suggests to me that the method does not account for such an array having (coordinate) variables that are not dimensions or a scalar.

What did you expect to happen?

I expect the MCVE below to return its input unmodified.

Minimal Complete Verifiable Example

```Python import xarray as xr

a = xr.DataArray([[0, 1], [2, 3]], {"x": [0, 1], "y": [0, 1]}) a = a.stack({"n": ("x", "y")}) a = a.chunk({"n": 2}) a.map_blocks(lambda x: x) ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python File ~/tmp/bug/venv/lib/python3.11/site-packages/xarray/core/dataarray.py:5526, in DataArray.map_blocks(self, func, args, kwargs, template) 5428 """ 5429 Apply a function to each block of this DataArray. 5430 (...) 5522 month (time) int64 dask.array<chunksize=(24,), meta=np.ndarray> 5523 """ 5524 from xarray.core.parallel import map_blocks -> 5526 return map_blocks(func, self, args, kwargs, template)

File ~/tmp/bug/venv/lib/python3.11/site-packages/xarray/core/parallel.py:539, in map_blocks(func, obj, args, kwargs, template) 535 for chunk_tuple in itertools.product(*ichunk.values()): 536 # mapping from dimension name to chunk index 537 chunk_index = dict(zip(ichunk.keys(), chunk_tuple)) --> 539 blocked_args = [ 540 subset_dataset_to_block(graph, gname, arg, input_chunk_bounds, chunk_index) 541 if isxr 542 else arg 543 for isxr, arg in zip(is_xarray, npargs) 544 ] 546 # raise nice error messages in _wrapper 547 expected: ExpectedDict = { 548 # input chunk 0 along a dimension maps to output chunk 0 along the same dimension 549 # even if length of dimension is changed by the applied function (...) 563 }, 564 }

File ~/tmp/bug/venv/lib/python3.11/site-packages/xarray/core/parallel.py:540, in <listcomp>(.0) 535 for chunk_tuple in itertools.product(*ichunk.values()): 536 # mapping from dimension name to chunk index 537 chunk_index = dict(zip(ichunk.keys(), chunk_tuple)) 539 blocked_args = [ --> 540 subset_dataset_to_block(graph, gname, arg, input_chunk_bounds, chunk_index) 541 if isxr 542 else arg 543 for isxr, arg in zip(is_xarray, npargs) 544 ] 546 # raise nice error messages in _wrapper 547 expected: ExpectedDict = { 548 # input chunk 0 along a dimension maps to output chunk 0 along the same dimension 549 # even if length of dimension is changed by the applied function (...) 563 }, 564 }

File ~/tmp/bug/venv/lib/python3.11/site-packages/xarray/core/parallel.py:195, in subset_dataset_to_block(graph, gname, dataset, input_chunk_bounds, chunk_index) 190 graph[chunk_variable_task] = ( 191 tuple, 192 [variable.dims, chunk, variable.attrs], 193 ) 194 else: --> 195 assert name in dataset.dims or variable.ndim == 0 197 # non-dask array possibly with dimensions chunked on other variables 198 # index into variable appropriately 199 subsetter = { 200 dim: _get_chunk_slicer(dim, chunk_index, input_chunk_bounds) 201 for dim in variable.dims 202 }

AssertionError:

```

Anything else we need to know?

No response

Environment

What happened?

After upgrading numpy from 1.23.5 to 1.24.1, calling the unstack method on an xarray.DataArray with integer data produces the warning <__array_function__ internals>:200: RuntimeWarning: invalid value encountered in cast. I think this relates to "ongoing work to improve the handling and promotion of dtypes" (Numpy 1.24.0 Release Notes), and is catching the fact that the method attempts to provide nan as a fill value on an integer array.

What did you expect to happen?

In the case below, where there is no need for a fill value, I do not expect to get a warning.

Minimal Complete Verifiable Example

```Python import xarray as xr import numpy as np

np.seterr(all='raise') # uncomment to convert warning to error

da = xr.DataArray( data=np.array([[0]], dtype=int), coords={'x': [0], 'y': [1]}, ) da = da.stack({'z': ['x', 'y']}) da.unstack() ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Python <__array_function__ internals>:200: RuntimeWarning: invalid value encountered in cast <xarray.DataArray (x: 1, y: 1)> array([[0]]) Coordinates: * x (x) int64 0 * y (y) int64 1

Anything else we need to know?

No response

Environment

Is your feature request related to a problem?

A scalar coordinate has the counter-intuitive property of being able to hop from one data variable to another. ``` import xarray as xr

a = xr.Dataset( data_vars={"a": (("x",), [0, 0])}, coords={ "x": [0.1, 2.3], "y": 42, }, ) b = xr.Dataset( data_vars={"b": ("x", [1, 1])}, coords={ "x": [0.1, 2.3], }, ) c = xr.merge((a, b)) Only `a` had the scalar coordinate `y` before merging, but now `c["b"]` has caught it: <xarray.DataArray 'b' (x: 2)> array([1, 1]) Coordinates: * x (x) float64 0.1 2.3 y int64 42 I think this is a bug in a way, because it does not reflect the NetCDF4 data model's ability to keep `y` as a coordinate on `a` alone. Note the "coordinates" attributes in the result of `c.to_netcdf`: netcdf c { dimensions: x = 2 ; variables: int64 a(x) ; a:coordinates = "y" ; double x(x) ; x:_FillValue = NaN ; int64 y ; int64 b(x) ; b:coordinates = "y" ; <---- Says who!? ```

Describe the solution you'd like

I would like each data variable in a dataset to keep track of its own scalar coordinates (as they can, of course and absolutely essentially, do for dimension coordinates). To continue the example above, I think c should have a representation that would lead to the following serialization: netcdf c { dimensions: x = 2 ; variables: int64 a(x) ; a:coordinates = "y" ; double x(x) ; x:_FillValue = NaN ; int64 y ; int64 b(x) ;

Describe alternatives you've considered

No response

Additional context

I think this feature could also help with #4501, wherein squeezing demotes a length-one non-dimensional coordinate to a scalar coordinate without tracking its own scalar coordinate. Egads, that's an ugly sentence. I'll elaborate over there.

Most importantly, this feature would solve a real problem I've encountered: model outputs, one for each combination of model parameters, that record parameters as a scalar coordinate only on the data variables the parameter affects. If you want to concatenate these together with XArray, you invariably get a lot of unncecessary data duplication. A contrived example with two outputs, in which the "temp" variable depends on parameter "time" but the "pressure" variable does not: output_42 = xr.Dataset({ "temp": xr.DataArray( data=[10, 9, 8], dims=("depth",), coords={ "depth": [0, 5, 10], "time": 42, }, ), "pressure": xr.DataArray( data=[0, 7, 14], coords={"depth": [0, 5, 10]}, ) }) output_88 = xr.Dataset({ "temp": xr.DataArray( data=[11, 10, 10], dims=("depth",), coords={ "depth": [0, 5, 10], "time": 88, }, ), "pressure": xr.DataArray( data=[0, 7, 14], coords={"depth": [0, 5, 10]}, ) }) I think it should be possible to concatenate these datasets without duplicating "pressure", like so: <xarray.Dataset> Dimensions: (depth: 3, time: 2) Coordinates: * depth (depth) int64 0 5 10 * time (time) int64 42 88 Data variables: temp (time, depth) int64 10 9 8 11 10 10 pressure (depth) int64 0 7 14 I can't get to there with any variation on xr.concat((output_42, output_88), dim="time", data_vars="minimal"), which I guess can be explained by the fact that "time" is associated with both "temp" and "pressure" in XArray's internal representation.

What is your issue?

Under the assumption that deterministic output is preferred whenever feasible, I'd like to point out that the variable names written into "coordinates" attributes with .to_netcdf are not ordered deterministically. For pipelines that depend on file hashes to validate dependencies, this can be a real headache.

Consider the dataset xarray.Dataset({"x": ((), 0)}, coords={"a": 0, "b": 0}). The NetCDF file XArray writes will include either: variables: int64 x ; x:coordinates = "a b" ; int64 a ; int64 b ; or variables: int64 x ; x:coordinates = "b a" ; int64 a ; int64 b ;

My review of _encode_coordinates leads me to think the behavior results from collecting names in a set. I'd be happy to offer a PR to make the coordinates attribute deterministic. I am not aware of a CF convention regarding any ordering, but would research and follow if it exists. If not, then I would probably sort at L701 and L722.

• [x] Closes #8026
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• ~~[ ] New functions/methods are listed in api.rst~~

The PR changes conventions._encode_coordinates to add sorted during the creation of "coordinates" strings de novo. It does not touch user-specified coordinates attributes or encodings.

The PR adds a test, but also changes two pre-existing tests that allowed for non-deterministic ordering of the coordinates string.

In reviewing the netCDF data model and CF convenstions, I confirmed that there is no requirement on the ordering of names in the coordinates attribute. I also learned that the global coordinates attribute created by XArray for non-dimension coordinates that are not associated with a variable is not compliant with CF conventions, as was discussed in 2014. In 2021, CF-Conventions 1.9 added "Domain Variables", which appear to provide a CF compliant way to handle this situation. I will likely open an enhancement issue to weigh making that change.

• [x] Closes #7250
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Just what @benbovy said to do in #7250. Thank you!

What happened?

The code example below results in False, because the data type of the a coordinate is changed from 'i4' to 'i8'.

What did you expect to happen?

I expect the result to be True. Creating a MultiIndex should not change the data type of the Indexes from which it is built.

Minimal Complete Verifiable Example

```Python import xarray as xr import numpy as np

ds = xr.Dataset(coords={'a': np.array([0], dtype='i4')}) ds['a'].values.dtype == ds.stack(b=('a',))['a'].values.dtype ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment