What is your issue?

We raise an error on grouping without supplying dims, but not for float indexes — is this intentional or an oversight?

This is without flox installed

```python

da = xr.tutorial.open_dataset("air_temperature")['air']

da.drop_vars('lat').groupby('lat').sum() ```

```

ValueError Traceback (most recent call last) Cell In[8], line 1 ----> 1 da.drop_vars('lat').groupby('lat').sum() ... ValueError: cannot reduce over dimensions ['lat']. expected either '...' to reduce over all dimensions or one or more of ('time', 'lon'). ```

But with a float index, we don't raise:

python da.groupby('lat').sum()

...returns the original array:

Out[15]: <xarray.DataArray 'air' (time: 2920, lat: 25, lon: 53)> array([[[296.29 , 296.79 , 297.1 , ..., 296.9 , 296.79 , 296.6 ], [295.9 , 296.19998, 296.79 , ..., 295.9 , 295.9 , 295.19998], [296.6 , 296.19998, 296.4 , ..., 295.4 , 295.1 , 294.69998], ...

And if we try this with a non-float index, we get the error again:

python da.groupby('time').sum()

ValueError: cannot reduce over dimensions ['time']. expected either '...' to reduce over all dimensions or one or more of ('lat', 'lon').

What is your issue?

There seems to be a common pattern for writing zarrs from a distributed set of machines, in parallel. It's somewhat described in the prose of the io docs. Quoting:

• Creating the template — "the first step is creating an initial Zarr store without writing all of its array data. This can be done by first creating a Dataset with dummy values stored in dask, and then calling to_zarr with compute=False to write only metadata to Zarr"
• Writing out each region from workers — "a Zarr store with the correct variable shapes and attributes exists that can be filled out by subsequent calls to to_zarr. The region provides a mapping from dimension names to Python slice objects indicating where the data should be written (in index space, not coordinate space)"

I've been using this fairly successfully recently. It's much better than writing hundreds or thousands of data variables, since many small data variables create a huge number of files.

Are there some tools we can provide to make this easier? Some ideas: - [ ] compute=False is arguably a less-than-obvious kwarg meaning "write metadata". Maybe this should be a method, maybe it's a candidate for renaming? Or maybe make_template can be an abstraction over it. Something like xarray_beam.make_template to make the template from a Dataset? - Or from an array of indexes? - https://github.com/pydata/xarray/issues/8343 - https://github.com/pydata/xarray/pull/8460 - [ ] What happens if one worker's data isn't aligned on some dimensions? Will that write to the wrong location? Could we offer an option, similar to the above, to reindex on the template dimensions?

• [ ] When writing a region, we need to drop other vars. Can we offer this as a kwarg? Occasionally I'll add a dimension with an index to a dataset, run the function to write it — and it'll fail, because I forgot to add that index to the .drop_vars call that precedes the write. When we're writing a template, all the indexes are written up front anyway. (edit: #6260)
  • https://github.com/pydata/xarray/pull/8460

More minor papercuts: - [ ] I've hit an issue where writing a region seemed to cause the worker to attempt to load the whole array into memory — can we offer guarantees for when (non-metadata) data will be loaded during to_zarr? - [ ] How about adding raise_if_dask_computes to our public API? The alternative I've been doing is watching htop and existing if I see memory ballooning, which is less cerebral... - [ ] It doesn't seem easy to write coords on a DataArray. For example, writing xr.tutorial.load_dataset('air_temperature').assign_coords(lat2=da.lat + 2, a=(('lon',), ['a'] * len(da.lon))).chunk().to_zarr('foo.zarr', compute=False) will cause the non-index coords to be written as empty. But writing them separately conflicts with having a single variable. Currently I manually load each coord before writing, which is not super-friendly.

Some things that were in the list here, as they've been completed!! - [x] Requiring region to be specified as an int range can be inconvenient — would it feasible to have a function that grabs the template metadata, calculates the region ints, and then calculates the implied indexes? - Edit: suggested at https://github.com/pydata/xarray/issues/7702

Is your feature request related to a problem?

We currently have query, which can runs a numexpr string using eval.

Describe the solution you'd like

Should we add an eval method itself? I find that when building something for the command line, allowing people to pass an eval-able expression can be a good interface.

Describe alternatives you've considered

No response

Additional context

No response

Is your feature request related to a problem?

Currently we need to know the order of dims when using region in to_zarr. Generally in xarray we're fine with the order, because we have the names, so this is a bit of an aberration. It means that code needs to carry around the correct order of dims.

Here's an MCVE:

```python

ds = xr.tutorial.load_dataset('air_temperature')

ds.to_zarr('foo', mode='w')

ds.transpose(..., 'lat').to_zarr('foo', mode='r+')

ValueError: variable 'air' already exists with different dimension names ('time', 'lat', 'lon') != ('time', 'lon', 'lat'), but changing variable dimensions is not supported by to_zarr().

```

Describe the solution you'd like

I think we should be able to transpose them based on the target?

Describe alternatives you've considered

No response

Additional context

No response

What happened?

.chunk's docstring states:

``` """Coerce this array's data into a dask arrays with the given chunks.

    If this variable is a non-dask array, it will be converted to dask
    array. If it's a dask array, it will be rechunked to the given chunk
    sizes.

```

...but this doesn't happen for 0 dim arrays; example below.

For context, as part of #8245, I had a function that creates a template array. It created an empty DataArray, then expanded dims for each dimension. And it kept blowing up memory! ...until I realized that it was actually not a lazy array.

What did you expect to happen?

It may be that we can't have a 0-dim dask array — but then we should raise in this method, rather than return the wrong thing.

Minimal Complete Verifiable Example

```Python [ins] In [1]: type(xr.DataArray().chunk().data) Out[1]: numpy.ndarray

[ins] In [2]: type(xr.DataArray(1).chunk().data) Out[2]: numpy.ndarray

[ins] In [3]: type(xr.DataArray([1]).chunk().data) Out[3]: dask.array.core.Array ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

What is your issue?

Our to_zarr methods on DataArray & Dataset are slightly inconsistent — Dataset.to_zarr has write_empty_chunks and chunkmanager_store_kwargs. They're also in a different order.

Up a level — not sure of the best way of enforcing consistency here; a couple of ideas. - We could have tests that operate on both a DataArray and Dataset, parameterized by fixtures (might also help reduce the duplication in some of our tests), though we then need to make the tests generic. We could have some general tests which just test that methods work, and then delegate to the current per-object tests for finer guarantees. - We could have a tool which collects the differences between DataArray & Dataset methods and snapshots them — then we'll see if they diverge, while allowing for some divergences.

Is your feature request related to a problem?

At the end of the dev call, we discussed ways to do better at maintenance. I'd like to make Xarray a wonderful place to contribute, partly because it was so formative for me in becoming more involved with software engineering.

Describe the solution you'd like

We've already come far, because of the hard work of many of us!

A few ideas, in increasing order of radical-ness - We looked at @andersy005's dashboards for PRs & Issues. Could we expose this, both to hold ourselves accountable and signal to potential contributors that we care about turnaround time for their contributions? - Is there a systematic way of understanding who should review something? - FWIW a few months ago I looked for a bot that would recommend a reviewer based on who had contributed code in the past, which I think I've seen before. But I couldn't find one generally available. This would be really helpful — we wouldn't have n people each assessing whether they're the best reviewer for each contribution. If anyone does better than me at finding something like this, that would be awesome. - Could we add a label so people can say "now I'm waiting for a review", and track how long those stay up? - Ensuring the 95th percentile is < 2 days is more important than the median being in the hours. It does pain me when I see PRs get dropped for a few weeks. TBC, I'm as responsible as anyone. - Could we have a bot that asks for feedback on the review process — i.e. "I received a prompt and helpful review", "I would recommend a friend contribute to Xarray", etc?

Describe alternatives you've considered

No response

Additional context

There's always a danger with making stats legible that Goodhart's law strikes. And sometimes stats are not joyful, and lots of people come here for joy. So probably there's a tradeoff.

What happened:

The new docs are looking great! One small issue — the lists don't appear as lists; e.g.

from https://xarray.pydata.org/en/latest/generated/xarray.Dataset.query.html

Do we need to change the rst convention?

What you expected to happen:

As bullets, with linebreaks

MCVE Code Sample

We have a very verbose warning in 0.15: it prints on every groupby on an object with multidimensional coords.

So the notebook I'm currently working on has red sections like: /home/mroos/.local/lib/python3.7/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims /home/mroos/.local/lib/python3.7/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims /home/mroos/.local/lib/python3.7/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims /home/mroos/.local/lib/python3.7/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims /home/mroos/.local/lib/python3.7/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims

Unless there's a way of reducing its verbosity (e.g. only print once per session?), let's aim to push the change through and remove the warning soon?

```python

Your code here

In [2]: import xarray as xr

In [4]: import numpy as np

In [16]: da = xr.DataArray(np.random.rand(2,3), dims=list('ab'))

In [17]: da = da.assign_coords(foo=(('a','b'),np.random.rand(2,3)))

In [18]: da.groupby('a').mean(...)
[...]/python3.6/site-packages/xarray/core/common.py:664: FutureWarning: This DataArray contains multi-dimensional coordinates. In the future, the dimension order of these coordinates will be restored as well unless you specify restore_coord_dims=False. self, group, squeeze=squeeze, restore_coord_dims=restore_coord_dims Out[18]: <xarray.DataArray (a: 2)> array([0.59216558, 0.58616892]) Dimensions without coordinates: a

```

Output of xr.show_versions()

This is out of date: http://xarray.pydata.org/en/stable/roadmap.html#current-core-developers

MCVE Code Sample

```python

In a notebook with html repr enabled

xr.Dataset({0: (('a','b'), np.random.rand(2,3))})

gives:

AttributeError Traceback (most recent call last) /j/office/app/research-python/conda/envs/2019.10/lib/python3.7/site-packages/IPython/core/formatters.py in call(self, obj) 343 method = get_real_method(obj, self.print_method) 344 if method is not None: --> 345 return method() 346 return None 347 else:

~/.local/lib/python3.7/site-packages/xarray/core/dataset.py in repr_html(self) 1632 if OPTIONS["display_style"] == "text": 1633 return f"

{escape(repr(self))}

~/.local/lib/python3.7/site-packages/xarray/core/formatting_html.py in dataset_repr(ds) 268 dim_section(ds), 269 coord_section(ds.coords), --> 270 datavar_section(ds.data_vars), 271 attr_section(ds.attrs), 272 ]

~/.local/lib/python3.7/site-packages/xarray/core/formatting_html.py in _mapping_section(mapping, name, details_func, max_items_collapse, enabled) 165 return collapsible_section( 166 name, --> 167 details=details_func(mapping), 168 n_items=n_items, 169 enabled=enabled,

~/.local/lib/python3.7/site-packages/xarray/core/formatting_html.py in summarize_vars(variables) 131 vars_li = "".join( 132 f"

~/.local/lib/python3.7/site-packages/xarray/core/formatting_html.py in <genexpr>(.0) 131 vars_li = "".join( 132 f"

~/.local/lib/python3.7/site-packages/xarray/core/formatting_html.py in summarize_variable(name, var, is_index, dtype, preview) 96 cssclass_idx = " class='xr-has-index'" if is_index else "" 97 dims_str = f"({', '.join(escape(dim) for dim in var.dims)})" ---> 98 name = escape(name) 99 dtype = dtype or escape(str(var.dtype)) 100

/j/office/app/research-python/conda/envs/2019.10/lib/python3.7/html/init.py in escape(s, quote) 17 translated. 18 """ ---> 19 s = s.replace("&", "&") # Must be done first! 20 s = s.replace("<", "<") 21 s = s.replace(">", ">")

AttributeError: 'int' object has no attribute 'replace'

<xarray.Dataset> Dimensions: (a: 2, b: 3) Dimensions without coordinates: a, b Data variables: 0 (a, b) float64 0.5327 0.927 0.8582 0.8825 0.9478 0.09475 ```

Problem Description

I think this may be an uncomplicated fix: coerce the keys to str

Output of xr.show_versions()

Note the different types in the final two outputs

``` python

periods = pd.period_range(start='2000', freq='B', periods=6000) np_points = np.random.rand(6000,20)

period_array = xr.DataArray(np_points) period_array['dim_0']=periods period_array

Out[87]: <xarray.DataArray (dim_0: 6000, dim_1: 20)> array([[ 0.36453381, 0.65939328, 0.65642922, ..., 0.66950028, 0.03690508, 0.85428786], [ 0.06142194, 0.6391667 , 0.93972185, ..., 0.26272683, 0.17446443, 0.05473016], [ 0.06888458, 0.88798184, 0.7004805 , ..., 0.54081794, 0.11690242, 0.71239621], ..., [ 0.46578244, 0.47498626, 0.11854992, ..., 0.73731368, 0.44784859, 0.24722402], [ 0.02694025, 0.26113875, 0.27635559, ..., 0.6397514 , 0.94297744, 0.50903873], [ 0.2302912 , 0.5255501 , 0.98877204, ..., 0.51659326, 0.5516555 , 0.10720623]]) Coordinates: * dim_0 (dim_0) object 2000-01-03 2000-01-04 2000-01-05 2000-01-06 ... * dim_1 (dim_1) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

period_array.dim_0.to_index() Out[93]: PeriodIndex(['2000-01-03', '2000-01-04', '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-10', '2000-01-11', '2000-01-12', '2000-01-13', '2000-01-14', ... '2022-12-19', '2022-12-20', '2022-12-21', '2022-12-22', '2022-12-23', '2022-12-26', '2022-12-27', '2022-12-28', '2022-12-29', '2022-12-30'], dtype='int64', name=u'dim_0', length=6000, freq='B') In [95]:

period_array.transpose('dim_0').dim_0.to_index() Out[95]: Index([2000-01-03, 2000-01-04, 2000-01-05, 2000-01-06, 2000-01-07, 2000-01-10, 2000-01-11, 2000-01-12, 2000-01-13, 2000-01-14, ... 2022-12-19, 2022-12-20, 2022-12-21, 2022-12-22, 2022-12-23, 2022-12-26, 2022-12-27, 2022-12-28, 2022-12-29, 2022-12-30], dtype='object', name=u'dim_0', length=6000)

```