What is your issue?

I looked a bit into the frequency string update & found 3 issues we could improve upon.

1. Apart from "M", pandas also deprecated "Y", and "Q", in favor of "YE" and "QE". (And they are discussing renaming "MS" to "MB"). Should we do the same?

2. Should we translate the new freq strings to the old ones if pandas < 2.2 is installed? Otherwise we get the following situation: python import xarray as xr xr.date_range("1600-02-01", periods=3, freq="M") # deprecation warning xr.date_range("1600-02-01", periods=3, freq="ME") # ValueError: Invalid frequency: ME

3. date_range_like can emit deprecation warnings without a way to mitigate them if pandas < 2.2 is installed. (When a DatetimeIndex) is passed. Could be nice to translate the old freq string to the new one without a warning.

I have played around with 2. and 3. and can open a PR if you are on board.

@spencerkclark @aulemahal

• pandas-dev/pandas#55792
• pandas-dev/pandas#55553
• pandas-dev/pandas#56840

What happened?

As per: https://github.com/pydata/xarray/pull/8246#discussion_r1340357405

Passing a negative step in a slice to select a non-chunked zarr-backed datasets raises an error.

What did you expect to happen?

zarr should allow negative step (probably?)

Minimal Complete Verifiable Example

```Python import xarray as xr

create a zarr dataset

air = xr.tutorial.open_dataset("air_temperature") air.to_zarr("test.zarr")

ds = xr.open_dataset("test.zarr", engine="zarr") ds.air[::-1, ].load()

note that this works if the dataset is backed by dask

ds_dask = xr.open_dataset("test.zarr", engine="zarr", chunks="auto") ds_dask.air[::-1, ].load() ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python File ~/code/xarray/xarray/core/parallelcompat.py:93, in guess_chunkmanager(manager) 91 if isinstance(manager, str): 92 if manager not in chunkmanagers: ---> 93 raise ValueError( 94 f"unrecognized chunk manager {manager} - must be one of: {list(chunkmanagers)}" 95 ) 97 return chunkmanagers[manager] 98 elif isinstance(manager, ChunkManagerEntrypoint): 99 # already a valid ChunkManager so just pass through

ValueError: unrecognized chunk manager dask - must be one of: [] ```

Anything else we need to know?

The error comes from https://github.com/zarr-developers/zarr-python/blob/6ec746ef1242dd9fec26b128cc0b3455d28ad6f0/zarr/indexing.py#L174 so it would need an upstream fix first.

cc @dcherian is this what you had in mind?

Environment

What happened?

The html repr of a cftime array wants to display the "calendar", which it cannot if it is empty.

What did you expect to happen?

No error.

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

data_obs = np.random.randn(3) time_obs = xr.date_range("2000-01-01", periods=3, freq="YS", calendar="noleap")

obs = xr.DataArray(data_obs, coords={"time": time_obs})

o = obs[:0]

xr.core.formatting_html.array_repr(o) ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python ValueError Traceback (most recent call last) Input In [1], in <cell line: 12>() 8 obs = xr.DataArray(data_obs, coords={"time": time_obs}) 10 o = obs[:0] ---> 12 xr.core.formatting_html.array_repr(o)

File ~/code/xarray/xarray/core/formatting_html.py:318, in array_repr(arr) 316 if hasattr(arr, "xindexes"): 317 indexes = _get_indexes_dict(arr.xindexes) --> 318 sections.append(index_section(indexes)) 320 sections.append(attr_section(arr.attrs)) 322 return _obj_repr(arr, header_components, sections)

File ~/code/xarray/xarray/core/formatting_html.py:195, in _mapping_section(mapping, name, details_func, max_items_collapse, expand_option_name, enabled) 188 expanded = _get_boolean_with_default( 189 expand_option_name, n_items < max_items_collapse 190 ) 191 collapsed = not expanded 193 return collapsible_section( 194 name, --> 195 details=details_func(mapping), 196 n_items=n_items, 197 enabled=enabled, 198 collapsed=collapsed, 199 )

File ~/code/xarray/xarray/core/formatting_html.py:155, in summarize_indexes(indexes) 154 def summarize_indexes(indexes): --> 155 indexes_li = "".join( 156 f"

{indexes_li}

File ~/code/xarray/xarray/core/formatting_html.py:156, in <genexpr>(.0) 154 def summarize_indexes(indexes): 155 indexes_li = "".join( --> 156 f"

{indexes_li}

File ~/code/xarray/xarray/core/formatting_html.py:140, in summarize_index(coord_names, index) 138 index_id = f"index-{uuid.uuid4()}" 139 preview = escape(inline_index_repr(index)) --> 140 details = short_index_repr_html(index) 142 data_icon = _icon("icon-database") 144 return ( 145 f"

File ~/code/xarray/xarray/core/formatting_html.py:132, in short_index_repr_html(index) 129 if hasattr(index, "repr_html"): 130 return index.repr_html() --> 132 return f"

{escape(repr(index))}

File ~/code/xarray/xarray/core/indexes.py:547, in PandasIndex.repr(self) 546 def repr(self): --> 547 return f"PandasIndex({repr(self.index)})"

File ~/code/xarray/xarray/coding/cftimeindex.py:353, in CFTimeIndex.repr(self) 345 end_str = format_times( 346 self.values[-REPR_ELLIPSIS_SHOW_ITEMS_FRONT_END:], 347 display_width, 348 offset=offset, 349 first_row_offset=offset, 350 ) 351 datastr = "\n".join([front_str, f"{' '*offset}...", end_str]) --> 353 attrs_str = format_attrs(self) 354 # oneliner only if smaller than display_width 355 full_repr_str = f"{klass_name}([{datastr}], {attrs_str})"

File ~/code/xarray/xarray/coding/cftimeindex.py:272, in format_attrs(index, separator) 267 def format_attrs(index, separator=", "): 268 """Format attributes of CFTimeIndex for repr.""" 269 attrs = { 270 "dtype": f"'{index.dtype}'", 271 "length": f"{len(index)}", --> 272 "calendar": f"'{index.calendar}'", 273 "freq": f"'{index.freq}'" if len(index) >= 3 else None, 274 } 276 attrs_str = [f"{k}={v}" for k, v in attrs.items()] 277 attrs_str = f"{separator}".join(attrs_str)

File ~/code/xarray/xarray/coding/cftimeindex.py:698, in CFTimeIndex.calendar(self) 695 """The calendar used by the datetimes in the index.""" 696 from .times import infer_calendar_name --> 698 return infer_calendar_name(self)

File ~/code/xarray/xarray/coding/times.py:374, in infer_calendar_name(dates) 371 return sample.calendar 373 # Error raise if dtype is neither datetime or "O", if cftime is not importable, and if element of 'O' dtype is not cftime. --> 374 raise ValueError("Array does not contain datetime objects.")

ValueError: Array does not contain datetime objects. ```

Anything else we need to know?

Bisected to 7379923de756a2bcc59044d548f8ab7a68b91d4e use _repr_inline_ for indexes that define it.

Environment

What happened?

da.xindexes or da.indexes raises an error when trying to generate the repr if there are no coords (indexes)

What did you expect to happen?

Displaying an empty Mappable?

Minimal Complete Verifiable Example

Python xr.DataArray([3, 5]).indexes xr.DataArray([3, 5]).xindexes

MVCE confirmation

• [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [x] Complete example — the example is self-contained, including all data and the text of any traceback.
• [x] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [x] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python Out[9]: --------------------------------------------------------------------------- ValueError Traceback (most recent call last) File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/core/formatters.py:708, in PlainTextFormatter.call(self, obj) 701 stream = StringIO() 702 printer = pretty.RepresentationPrinter(stream, self.verbose, 703 self.max_width, self.newline, 704 max_seq_length=self.max_seq_length, 705 singleton_pprinters=self.singleton_printers, 706 type_pprinters=self.type_printers, 707 deferred_pprinters=self.deferred_printers) --> 708 printer.pretty(obj) 709 printer.flush() 710 return stream.getvalue()

File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/lib/pretty.py:410, in RepresentationPrinter.pretty(self, obj) 407 return meth(obj, self, cycle) 408 if cls is not object \ 409 and callable(cls.dict.get('repr')): --> 410 return _repr_pprint(obj, self, cycle) 412 return _default_pprint(obj, self, cycle) 413 finally:

File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/lib/pretty.py:778, in repr_pprint(obj, p, cycle) 776 """A pprint that just redirects to the normal repr function.""" 777 # Find newlines and replace them with p.break() --> 778 output = repr(obj) 779 lines = output.splitlines() 780 with p.group():

File ~/code/xarray/xarray/core/indexes.py:1659, in Indexes.repr(self) 1657 def repr(self): 1658 indexes = formatting._get_indexes_dict(self) -> 1659 return formatting.indexes_repr(indexes)

File ~/code/xarray/xarray/core/formatting.py:474, in indexes_repr(indexes, max_rows) 473 def indexes_repr(indexes, max_rows: int | None = None) -> str: --> 474 col_width = _calculate_col_width(chain.from_iterable(indexes)) 476 return _mapping_repr( 477 indexes, 478 "Indexes", (...) 482 max_rows=max_rows, 483 )

File ~/code/xarray/xarray/core/formatting.py:341, in _calculate_col_width(col_items) 340 def _calculate_col_width(col_items): --> 341 max_name_length = max(len(str(s)) for s in col_items) if col_items else 0 342 col_width = max(max_name_length, 7) + 6 343 return col_width

ValueError: max() arg is an empty sequence ```

Anything else we need to know?

No response

Environment

I really should not open this can of worms but per https://github.com/pydata/xarray/issues/4450#issuecomment-697507489:

I'm always confused about whether ds.groupby(..., keep_attrs=True).mean() or ds.groupby(...).mean(keep_attrs=True) is correct. (similarly for rolling, coarsen etc.)

Also as I try to fix the keep_attr behavior in #4510 it would be good to know where they should go. So I tried to figure out how this is currently handled and found the following:

ds.xxx(keep_attrs=True).yyy() - all fixed

ds.xxx().yyy(keep_attrs=True) - coarsen (fixed in #5227) - groupby - groupby_bin - resample - rolling (adjusted in #4510) - rolling_exp (fixed in #4592) - weighted

So the working consensus seems to be to to ds.xxx().yyy(keep_attrs=True) - any comments on that?

(Edit: looking at this it is only half as bad, "only" coarsen, rolling (#4510), and rolling_exp would need to be fixed.)

Detailed analysis

edit: moved rolling after #4510, moved rolling_exp after #4592 and coarsen after #5227

What happened?

I tried a DataArray with more than one dimension coordinate. Unfortunately xr.align fails, which disallows any arithmetic operation - even when the coords are exactly the same.

What did you expect to happen?

No response

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

data = np.arange(12).reshape(3, 4)

y = [10, 20, 30] s = ["a", "b", "c"]

x = [1, 2, 3, 4]

da = xr.DataArray(data, dims=("y", "x"), coords={"x": x, "y": y, "s": ("y", s)}) da = da.set_xindex("s")

xr.align(da, da.y) # errors da + da # errors da + da.x # errors ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [ ] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python

ValueError Traceback (most recent call last) /home/mathause/code/mesmer/devel/prepare_for_surfer.ipynb Cell 28 line 1 12 da = xr.DataArray(data, dims=("y", "x"), coords={"x": x, "y": y, "s": ("y", s)}) 13 da = da.set_xindex("s") ---> 15 xr.align(da, da.y) # errors 17 da + da.x # errors

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:888, in align(join, copy, indexes, exclude, fill_value, *objects) 692 """ 693 Given any number of Dataset and/or DataArray objects, returns new 694 objects with aligned indexes and dimension sizes. ref='~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:0'>0;32m (...) 878 879 """ 880 aligner = Aligner( 881 objects, 882 join=join, ref='~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:0'>0;32m (...) 886 fill_value=fill_value, 887 ) --> 888 aligner.align() 889 return aligner.results

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:573, in Aligner.align(self) 571 self.find_matching_indexes() 572 self.find_matching_unindexed_dims() --> 573 self.assert_no_index_conflict() 574 self.align_indexes() 575 self.assert_unindexed_dim_sizes_equal()

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:318, in Aligner.assert_no_index_conflict(self) 314 if dup: 315 items_msg = ", ".join( 316 f"{k!r} ({v} conflicting indexes)" for k, v in dup.items() 317 ) --> 318 raise ValueError( 319 "cannot re-index or align objects with conflicting indexes found for " 320 f"the following {msg}: {items_msg}\n" 321 "Conflicting indexes may occur when\n" 322 "- they relate to different sets of coordinate and/or dimension names\n" 323 "- they don't have the same type\n" 324 "- they may be used to reindex data along common dimensions" 325 )

ValueError: cannot re-index or align objects with conflicting indexes found for the following dimensions: 'y' (2 conflicting indexes) Conflicting indexes may occur when - they relate to different sets of coordinate and/or dimension names - they don't have the same type - they may be used to reindex data along common dimensions ```

Anything else we need to know?

No response

Environment

What happened?

Running an in-memory groupby operation took much longer than expected. Turning off flox fixed this - but I don't think that's the idea ;-)

What did you expect to happen?

flox to be at least on par with our naive implementation

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

arr = np.random.randn(10, 10, 36530) time = xr.date_range("2000", periods=30365, calendar="noleap") da = xr.DataArray(arr, dims=("y", "x", "time"), coords={"time": time})

using max

print("max:") xr.set_options(use_flox=True) %timeit da.groupby("time.year").max("time") %timeit da.groupby("time.year").max("time", engine="flox")

xr.set_options(use_flox=False) %timeit da.groupby("time.year").max("time")

as reference

%timeit [da.sel(time=str(year)).max("time") for year in range(2000, 2030)]

using mean

print("mean:") xr.set_options(use_flox=True) %timeit da.groupby("time.year").mean("time") %timeit da.groupby("time.year").mean("time", engine="flox")

xr.set_options(use_flox=False) %timeit da.groupby("time.year").mean("time")

as reference

%timeit [da.sel(time=str(year)).mean("time") for year in range(2000, 2030)] ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python max: 158 ms ± 4.41 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 28.1 ms ± 318 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) 11.5 ms ± 52.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

mean: 95.6 ms ± 10.8 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) 34.8 ms ± 2.88 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) 15.2 ms ± 232 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) ```

Anything else we need to know?

No response

Environment

When I manually specify the units attribute for time, and then save the Dataset to_netcdf the string is truncated. See exaple

import pandas as pd
import xarray as xr

time = pd.date_range('2000-01-01', '2000-01-31', freq='6h')
ds = xr.Dataset(coords=dict(time=time))

units = 'days since 1975-01-01 00:00:00'
calendar = 'gregorian'
encoding=dict(time=dict(units=units, calendar=calendar))

ds.to_netcdf('test.nc', format='NETCDF4_CLASSIC', encoding=encoding)

! ncdump -h test.nc
# time:units = "days since 1975-01-01" ;

Some programs seem to require the hours to be present to interpret the time properly (e.g. panoply). When specifying the hour, a 'T' is added.

units = 'days since 1975-01-01 01:00:00'

! ncdump -h test.nc
# time:units = "days since 1975-01-01T01:00:00" ;

When xarray defines the time.units it works fine.

ds = xr.Dataset(coords=dict(time=time))
ds.to_netcdf('test.nc', format='NETCDF4_CLASSIC',)

! ncdump -h test.nc
# time:units = "hours since 2000-01-01 00:00:00" ;

xarray version 0.9.6

We are currently not testing with the newest version of netCDF4 because it is incompatible with pynio (the newest version is 1.5.4, we are at 1.5.3). This is unlikely to be fixed, see conda-forge/pynio-feedstock#90.

Therefore we need to think how to setup the tests so we use the newest version of netCDF4. Maybe just remove it from py38.yml?

And long term what to do with the pynio backend? Deprecate? Move to an external repo?

What happened?

xr.core.dataset.DataVariables assumes that everything that is in ds._dataset._variables and not in self._dataset._coord_names is a "data variable". However, since the index refactor we can end up with more _coord_names than _variables which breaks a number of stuff (e.g. the repr).

What did you expect to happen?

Well it seems this assumption is now wrong.

Minimal Complete Verifiable Example

Python ds = xr.Dataset(coords={"a": ("x", [1, 2, 3]), "b": ("x", ['a', 'b', 'c'])}) ds.set_index(z=['a', 'b']).reset_index("z", drop=True)

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Python ValueError: __len__() should return >= 0

Anything else we need to know?

The error comes from here

https://github.com/pydata/xarray/blob/63ba862d03c8d0cd8b44d2071bc360e9fed4519d/xarray/core/dataset.py#L368

Bisected to #5692 - which probably does not help too much.

Environment

This started as a question and I add it as reference. Maybe you have a comment.

There are several ways to calculate time series of seasonal data (starting from monthly or daily data):

```

load libraries

import pandas as pd import matplotlib.pyplot import numpy as np import xarray as xr

Create Example Dataset

time = pd.date_range('2000.01.01', '2010.12.31', freq='M') data = np.random.rand(*time.shape) ds = xr.DataArray(data, coords=dict(time=time))

(1) using resample

ds_res = ds.resample('Q-FEB', 'time') ds_res = ds_res.sel(time=ds_res['time.month'] == 2) ds_res = ds_res.groupby('time.year').mean('time')

(2) this is wrong

ds_season = ds.where(ds['time.season'] == 'DJF').groupby('time.year').mean('time')

(3) using where and rolling

mask other months with nan

ds_DJF = ds.where(ds['time.season'] == 'DJF')

rolling mean -> only Jan is not nan

however, we loose Jan/ Feb in the first year and Dec in the last

ds_DJF = ds_DJF.rolling(min_periods=3, center=True, time=3).mean()

make annual mean

ds_DJF = ds_DJF.groupby('time.year').mean('time')

ds_res.plot(marker='*') ds_season.plot() ds_DJF.plot()

plt.show() ```

(1) The first is to use resample with 'Q-FEB' as argument. This works fine. It does include Jan/ Feb in the first year, and Dec in the last year + 1. If this makes sense can be debated. One case where this does not work is when you have, say, two regions in your data set, for one you want to calculate DJF and for the other you want NovDecJan.

(2) Using 'time.season' is wrong as it combines Jan, Feb and Dec from the same year.

(3) The third uses where and rolling and you lose 'incomplete' seasons. If you replace ds.where(ds['time.season'] == 'DJF') with ds.groupby('time.month').where(summer_months), where summer_months is a boolean array it works also for non-standard 'summers' (or seasons) across the globe.

bottleneck cannot be built with python 3.10. See https://github.com/pydata/xarray/actions/runs/1731371015

pandas deprecates the method keyword in Index.get_loc, see pandas-dev/pandas#42269. Therefore we end up with about 5000 warnings in our upstream tests:

FutureWarning: Passing method to Index.get_loc is deprecated and will raise in a future version. Use index.get_indexer([item], method=...) instead

We should fix this before pandas releases because the warning will not be silent (FutureWarning) or ask pandas to give us more time and use a DeprecationWarning at the moment.

@benbovy @spencerkclark

```python import xarray as xr import numpy as np

ds = xr.DataArray([np.NaN, np.NaN])

ds.mean() ds.sum()

```

Problem description

ds.mean() returns NaN, ds.sum() returns 0. This comes from numpy (cp np.nanmean vs. np.nansum), so it might have to be discussed upstream, but I wanted to ask the xarray community first on their opinion. This is also relevant for #422 (what happens if the all weights are NaN or sum up to 0).

Expected Output

I would expect both to return np.nan.

Output of xr.show_versions()

I disabled pytest-xdist because a dask issue renders our CI unusable. As soon as dask releases a new version we should revert #6077 again.

What happened:

I could not open the test dataset on master. It's a bit strange that this is not picked up by the tests, so probably something to do with the environment I have (I just updated all packages).

@alexamici @aurghs does that tell you anything?

I can also try to figure it out.

Minimal Complete Verifiable Example:

calling open_dataset with "" is enough to trigger the error:

python import xarray as xr air = xr.open_dataset("")

Anything else we need to know?:

And the traceback:

Environment:

and my conda list:

edit: added the traceback

We currently pin netCDF4 to version 1.5. However, I think netCDF4 does not really follow semantic versioning, e.g. python 2 support was dropped in version 1.5.6. So they may actually be doing something like 1.major.minor[.patch] - I asked about their versioning scheme in Unidata/netcdf4-python#1090.

So I wonder if we would need to pin netCDF to version to version 1.5.4.

The last 5 days our Upstream CI failed silently with a timeout after 6h:

https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml?query=branch%3Amaster+event%3Aschedule

This was probably caused by #4934. As mentioned in dask/dask#4934 this is probably dask/dask#6738 which was merged 5 days ago.

We currently run mypy in the pre-commit hooks CI. However, this is done in an environment where no dependencies are installed. Therefore we missed the errors that pop up when running mypy with numpy 1.20 installed. (Please correct my if I misunderstood this). Should we add a new step to our CI and run mypy?

I think we should at least add this to ubuntu-latest py3.8. For more complete checks we could also go for ubuntu-latest py37-min-all-deps and upstream-dev.

h5py version 3.1 broke our tests. I pinned it to version 2.10 in #4569. We should therefore

• fix the issues
• unpin h5py again

The failures could be related to a change how strings are read: https://docs.h5py.org/en/latest/strings.html I am not sure if this has to be fixed in xarray or in h5necdf. I'd be happy if someone else took this one.

Failed tests:

Example failure:

```python traceback

assert_allclose(original, computed) E AssertionError: Left and right Dataset objects are not close E
E Differing coordinates: E L * dim3 (dim3) <U1 'a' 'b' 'c' 'd' 'e' 'f' 'g' 'h' 'i' 'j' E R * dim3 (dim3) object b'a' b'b' b'c' b'd' b'e' b'f' b'g' b'h' b'i' b'j'< ```

What happened:

Our doctests fail since numpy 1.20 came out:

https://github.com/pydata/xarray/pull/4760/checks?check_run_id=1818512841#step:8:69

What you expected to happen:

They don't ;-)

Minimal Complete Verifiable Example:

The following fails with numpy 1.20 while it converted np.NaN to an integer before (xarray.DataArray.pad at the bottom)

```python import numpy as np

x = np.arange(10) x = np.pad(x, 1, "constant", constant_values=np.nan) ``` requires numpy 1.20

Anything else we need to know?:

• that's probably related to https://numpy.org/doc/stable/release/1.20.0-notes.html#numpy-scalars-are-cast-when-assigned-to-arrays
• I asked if this behavior will stay: https://github.com/numpy/numpy/issues/16499#issuecomment-772342087
• One possibility is to add a check np.can_cast(constant_values.dtype, array.dtype) (or similar) for a better error message.

Would be nice if it did :-)

e.g. https://github.com/pydata/xarray/runs/1698988952

What happened:

I try to save the result of ds.resample("time": "Q-FEB").mean() with a 360_day calendar as netcdf. Thus, my first date is cftime.Datetime360Day(2000-02-30) (note the day). xarray then tries to use units='days since 2000-02-30' which fails with ValueError: day is out of range for month.

What you expected to happen:

The dataset can be saved.

Minimal Complete Verifiable Example:

```python import cftime import xarray as xr

time = xr.cftime_range("2000-02-30", "2001-01-01", freq="3M", calendar="360_day")

dates = np.asarray(time) reference_date = xr.coding.times.infer_datetime_units(dates)

'days since 2000-02-30 00:00:00.000000'

xr.coding.times.encode_cf_datetime(time)

ValueError

``` Traceback:

Anything else we need to know?:

This goes wrong here: https://github.com/pydata/xarray/blob/15537497136345ed67e9e8b089bcd4573df0b2ea/xarray/coding/times.py#L249-L250

A possible fix is to add the following lines:

python try: cftime._cftime._dateparse(reference_date) except ValueError: reference_date = type(dates[0])(dates[0].year, dates[0].month, 1) reference_date = xr.coding.times.format_cftime_datetime(reference_date)

To workaround set the encoding manually:

python encoding = {} encoding['time'] = {'units': 'days since 1850-01-01'} ds.to_netcdf(filename, encoding=encoding)

The windows tests seem to regularly time out after 60 min, which is annoying. All other tests seem to run about twice as fast... There is some stuff we can try (which may or may not help):

• [ ] switch to the newest windows image
• [ ] use mamba to resolve dependencies
• [ ] use matplotlib-base
• [ ] find the slowest tests using py.test xarray/tests/ --durations=100
• [ ] de-parametrize tests with a slow setup (if possible)
• [ ] reduce the number of xfails
• [ ] other ideas?

The Linux py36-bare-minimum will likely fail with a TypeError: 'NoneType' object is not callable due to python/importlib_metadata#270

This could be fixed by adding typing_extensions to the CI or maybe they fix that upstream.

We use mamba in the new CI:

https://github.com/pydata/xarray/blob/7152b41fa80a56db0ce88b241fbe4092473cfcf0/.github/workflows/upstream-dev-ci.yaml#L39

I just tested it and it is awesomely fast! However, it does not enforce strict channel priority, so packages from main get mixed in (see e.g. https://github.com/pydata/xarray/runs/1487637678#step:5:112 & search for main) & we don't want that. So I'd suggest to do:

bash mamba env update --override-channels -c conda-forge -f ci/requirements/py38.yml

(note there is a --strict-channel-priority option in mamba but it still installed packages from main when I tested it)

cc @andersy005: mybe you could add this to #4604?

See

(http://xarray.pydata.org/en/latest/generated/xarray.concat.html)

Can this be fixed? Or do we actually want to show both?

As discussed in the call today it would be nice to remove the upstream test from our ci and setup a "nightly" job to test against the development versions of pandas etc. This should lead to less failures unrelated to specific PRs. Question: how are we going to see that the upstream build failed? Having the failure in random PRs is not clean but very visible...

This could be done via github actions. See xpublish for an example setup: https://github.com/xarray-contrib/xpublish/tree/master/.github/workflows

xref #4313

Now that #4502 was fixed upstream we get a new issue in py38-upstream-dev (which I am pretty sure is unrelated as dask is not involved). xarray/tests/test_cftimeindex_resample.py::test_resample fails with:

python-traceback AttributeError: 'CFTimeIndex' object has no attribute '_id'

See: https://dev.azure.com/xarray/xarray/_build/results?buildId=4038&view=logs&jobId=603f3fdc-8af6-5e0a-f594-fa71bc949352&j=603f3fdc-8af6-5e0a-f594-fa71bc949352&t=51624cf6-228d-5319-1d6f-8cd30bcca2e7

The failure did not happen on the 14. Oct. (https://github.com/pydata/xarray/commit/db4f03e467d13229512f8f7924dc142db1b9486b the failures here are #4502) but it appeared on the 15. Oct. (https://github.com/pydata/xarray/commit/15537497136345ed67e9e8b089bcd4573df0b2ea)

Maybe a change in pandas? I have not looked at it closely - maybe @spencerkclark sees what's going on?

See ecmwf/cfgrib#167. This causes our doc build to fail (#4520). The tests do not fail because import cfgrib issues a ModuleNotFoundError therfore @requires_cfgrib skips the test...

todo - [ ] unpin docs again - [ ] it's probably quite a corner case but - should the tests fail? How would this be achieved...?

The DataArray tests are skipped in py38-all-but-dask - I have no idea why...

See

https://dev.azure.com/xarray/xarray/_build/results?buildId=4028&view=logs&j=15c5a299-0936-5372-ef76-f6800a28dcef&t=b8653be4-abae-5992-d733-6af037900ab6&l=108

test_dataarray.py should be just before test_dataset.py

Our upstream tests fail due to a change in dask. The likely culprit is dask/dask#6680 - dask.array.zeros_like does now take the meta keyword and thus it tries to coerce str to a bool which fails (and it didn't do before). The following should trigger the error:

```python import xarray as xr import dask

da = xr.DataArray(dask.array.array("")) xr.zeros_like(da, dtype=bool) `` Note thatzeros_likeis called inisnull` (which triggered the test failures):

https://github.com/pydata/xarray/blob/080caf4246fe2f4d6aa0c5dcb65a99b376fa669b/xarray/core/duck_array_ops.py#L100-L102

What happened:

```python /home/vsts/work/1/s/xarray/tests/test_duck_array_ops.py:499:

/home/vsts/work/1/s/xarray/testing.py:139: in compat_variable return a.dims == b.dims and (a._data is b._data or equiv(a.data, b.data)) /home/vsts/work/1/s/xarray/testing.py:36: in _data_allclose_or_equiv return duck_array_ops.array_equiv(arr1, arr2) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:246: in array_equiv flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2)) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:104: in isnull return zeros_like(data, dtype=bool) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:56: in f return wrapped(args, *kwargs) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/creation.py:174: in zeros_like return zeros( /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/wrap.py:78: in wrap_func_shape_as_first_arg return Array(dsk, name, chunks, dtype=dtype, meta=kwargs.get("meta", None)) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/core.py:1083: in new meta = meta_from_array(meta, dtype=dtype)

x = array('', dtype='<U1'), ndim = 0, dtype = dtype('bool')

def meta_from_array(x, ndim=None, dtype=None):
    """Normalize an array to appropriate meta object

    Parameters
    ----------
    x: array-like, callable
        Either an object that looks sufficiently like a Numpy array,
        or a callable that accepts shape and dtype keywords

    if dtype and meta.dtype != dtype:

      meta = meta.astype(dtype)

E ValueError: invalid literal for int() with base 10: ''

```

MCVE Code Sample

```python import xarray as xr import numpy as np

ds = xr.DataArray([1, 2], attrs=dict(tst="DataArray")).to_dataset(name="data") ds.attrs["tst"] = "Dataset" ds.map(np.mean, keep_attrs=True).data ```

returns python <xarray.DataArray 'data' ()> array(1.5)

Expected Output

python <xarray.DataArray 'data' ()> array(1.5) Attributes: tst: DataArray

Problem Description

Applying xr.Dataset.map(..., keep_attrs=True) does not retain the attributes of the DataArrays. Should it?

In constrast ds.mean(keep_attrs=True) retains DataArray-level attrs.

EDIT: corrected example

Output of xr.show_versions()

Matplotlib 3.3.1 broke the doc build so I pinned it to 3.3.0 - see #4340

The likely culprit is https://github.com/matplotlib/matplotlib/issues/18254 - thanks @jthielen for finding the matplotlib issue

What happened: da.sum(min_count=1) returns a TypeError if da has an integer dtype. Of course min_count is not necessary for integer data as it cannot contain NaN.

What you expected to happen: min_count should be ignored

Minimal Complete Verifiable Example:

python import xarray as xr da = xr.DataArray([[1, 2, 3], [4, 5, 6]]) da.sum(min_count=1)

Anything else we need to know?:

Full traceback

Environment:

Another follow up from #4060: quantile now emits a FutureWarning:

Minimal Complete Verifiable Example:

python xr.DataArray([1, 2, 3]).quantile(q=0.5)

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/variable.py:1866: FutureWarning: ``output_sizes`` should be given in the ``dask_gufunc_kwargs`` parameter. It will be removed as direct parameter in a future version. kwargs={"q": q, "axis": axis, "interpolation": interpolation},

We should probably check the warnings in the test suite - there may be others.

I tried to use apply_ufunc for a function that takes input of unequal length and requires vectorize=True which resulted in a ValueError. I think the problem stems from the way np.vectorize is called.

MCVE Code Sample

```python import xarray as xr import scipy as sp import scipy.stats import numpy as np

create dataarrays of unequal length

ds = xr.tutorial.open_dataset("air_temperature")

da1 = ds.air da2 = ds.air.isel(time=slice(None, 50))

function that takes arguments of unequal length and requires vectorizing

def mannwhitneyu(x, y): _, p = sp.stats.mannwhitneyu(x, y) return p

test that the function takes arguments of unequal length

mannwhitneyu(da1.isel(lat=0, lon=0), da2.isel(lat=0, lon=0))

xr.apply_ufunc( mannwhitneyu, da1, da2, input_core_dims=[["time"], ["time"]], exclude_dims=set(["time"]), vectorize=True, )

```

Returns

python ValueError: inconsistent size for core dimension 'n': 50 vs 2920 Note: the error stems from numpy.

Expected Output

A DataArray.

Problem Description

I can reproduce the problem in pure numpy:

python vec_wrong = np.vectorize(mannwhitneyu, signature="(n),(n)->()", otypes=[np.float]) vec_wrong(da1.values.T, da2.values.T)

The correct result is returned when the signature is changed: python vec_correct = np.vectorize(mannwhitneyu, signature="(m),(n)->()", otypes=[np.float]) vec_correct(da1.values.T, da2.values.T) So I think the signature needs to be changed when exclude_dims are present.

Versions

Is your feature request related to a problem? Please describe.

sum with min_count errors when passing more than one dim:

python import xarray as xr da = xr.DataArray([[1., 2, 3], [4, 5, 6]]) da.sum(["dim_0", "dim_1"], min_count=1)

Describe the solution you'd like The logic to calculate the number of valid elements is here: https://github.com/pydata/xarray/blob/1be777fe725a85b8cc0f65a2bc41f4bc2ba18043/xarray/core/nanops.py#L35

I think this can be fixed by replacing

mask.shape[axis] with np.take(a.shape, axis).prod()

Additional context Potentially relevant for #4351

It would be nice to be able to use a level of a MultiIndex for plotting.

MCVE Code Sample

```python import numpy as np import xarray as xr

da = xr.DataArray( np.random.randn(10), dims="x", coords=dict( a=("x", np.arange(10, 20)), b=("x", np.arange(1, 11)) ) )

da = da.set_index(x=["a", "b"]) da This creates the following DataArraypython <xarray.DataArray (x: 10)> array([-1.34516338, 0.97644817, -0.24032189, -0.70112418, -0.8686898 , -0.55607078, 0.56618151, 1.62847463, 0.84947296, -0.5775504 ]) Coordinates: * x (x) MultiIndex - a (x) int64 10 11 12 13 14 15 16 17 18 19 - b (x) int64 1 2 3 4 5 6 7 8 9 10 ```

Is there a way to plot a line using one of the levels of the MultiIindex? python da.plot(x="a") returns python ValueError: x must be either None or one of ('x')

python da.plot() returns

python TypeError: Plotting requires coordinates to be numeric or dates of type np.datetime64, datetime.datetime, cftime.datetime or pd.Interval. (which makes sense). If da is a 2D Variable the error is python ValueError: x and y must be coordinate variables

Expected Output

A line plot

Versions

MCVE Code Sample

```python import numpy as np import xarray as xr

dta = xr.DataArray([1., 1., 1.]) wgt = xr.DataArray(np.array([1, 1, 0], dtype=np.bool))

dta.weighted(wgt).mean() ``` Returns

<xarray.DataArray ()> array(2.)

Expected Output

<xarray.DataArray ()> array(1.)

Problem Description

Passing a boolean array as weights to the weighted mean returns the wrong result because the weights are not properly normalized (in this case). Internally the sum_of_weights is calculated as

python xr.dot(dta.notnull(), wgt) i.e. the dot product of two boolean arrays. This yields: <xarray.DataArray ()> array(True)

We'll need to convert it to int or float: python xr.dot(dta.notnull(), wgt * 1) which is correct <xarray.DataArray ()> array(2)

Versions

MCVE Code Sample

Add arbitrary change to the file doc/pandas.rst

bash git add doc/pandas.rst git commit -m "test"

The pre-commit hook will fail.

Expected Output

the pre-commit hook to pass

Problem Description

running isort -rc doc/* will change the following files: bash modified: contributing.rst modified: howdoi.rst modified: internals.rst modified: io.rst modified: pandas.rst modified: quick-overview.rst unfortunately it does not behave properly and deletes/ changes arbitrary lines. Can the pre-commit hook be told to only run on *.py files? On the command line this would be isort -rc *.py

MCVE Code Sample

```python import numpy as np import xarray as xr

data = np.random.randn(10, 10) data = np.abs(data)

da = xr.DataArray(data)

returns a diverging colormap

da.plot(vmax=2, center=0, extend="both")

returns a sequential colormap

da.plot(levels=[-2, -1, 0, 1, 2], extend="both")

```

Expected Output

A diverging colormap, maybe?

Problem Description

I was surprised by getting the viridis colormap until I realised that my data must all be positive and the colormap is infered from the data and not from levels. However, when specifying the range via vmax=2, center=0 it is not inferred from the data.

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import xarray as xr import numpy as np x = np.arange(4.) ds = xr.DataArray(x, dims=('dim', )) ds.rolling(dim=3, center=True).mean()

RESULT: array([nan, 1., 2., nan])

ds.rolling(dim=3, center=True).construct('window').mean('window')

RESULT: array([0.5, 1. , 2. , 2.5])

```

Problem description

ds.rolling(...).mean() and ds.rolling(...).construct().mean() yields different results. Because mean does skipna=True per default.

Expected Output

I would expect both ways to yield the same result.

Output of xr.show_versions()

There is a bunch of operations that need to be conducted on a 1D array. It could be handy to have a convinience method that mimicks numpys flatten, i.e. works along the lines:

def flatten(dataarray, name='stacked'): return dataarray.stack(**{name: dataarray.dims})

What do you think?

It would be nice to be able to do ds.average() to compute weighted averages (e.g. for geo data). Of course this would require the axes to be in a predictable order. Or to give a weight per dimension...

Seaborn recently dropped the apionly entry point (in version 0.10.0).

There is a dedicated import_seaborn function https://github.com/pydata/xarray/blob/47a6fc0c9b796997996d79d32358585d08021b9c/xarray/plot/utils.py#L24

but it is not used in one place

https://github.com/pydata/xarray/blob/47a6fc0c9b796997996d79d32358585d08021b9c/xarray/plot/utils.py#L146

which now causes test failures

https://dev.azure.com/xarray/xarray/_build/results?buildId=2111&view=ms.vss-test-web.build-test-results-tab&runId=29632&paneView=debug&resultId=108176

Todo: double check if the supported versions of seaborn still require this workaround.

MCVE Code Sample

```python import xarray as xr import numpy as np

d1 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[0, 1, 2, 3])) d2 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[0, 1, 2, 3]))

note: different coords

d3 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[1, 2, 3, 4]))

(d1 * d2).sum() # -> array(14) xr.dot(d1, d2) # -> array(14)

(d2 * d3).sum() # -> array(8) xr.dot(d2, d3) # -> ValueError

```

Expected Output

python <xarray.DataArray ()> array(8)

Problem Description

The last statement results in an python ValueError: indexes along dimension 'a' are not equal because xr.apply_ufunc defaults to join='exact'. However, I think this should work - but maybe there is a good reason for this to fail?

This is a problem for #2922 (weighted operations) - I think it is fine for the weights and data to not align.

Fixing this may be as easy as specifying join='inner' in

https://github.com/pydata/xarray/blob/e0fd48052dbda34ee35d2491e4fe856495c9621b/xarray/core/computation.py#L1181-L1187

@fujiisoup

Output of xr.show_versions()

MCVE Code Sample

``` python import numpy as np import xarray as xr time = xr.cftime_range("2006-01-01", periods=2, calendar="360_day")

da = xr.DataArray(time, dims=["time"]) da.encoding["dtype"] = np.float da.to_netcdf("tst.nc", format="NETCDF4_CLASSIC")

ds = xr.open_dataset("tst.nc") ds.to_netcdf("tst2.nc", format="NETCDF4_CLASSIC") ```

yields: python ValueError: could not safely cast array from dtype int64 to int32

Or an example without to_netcdf:

```python import numpy as np import xarray as xr

time = xr.cftime_range("2006-01-01", periods=2, calendar="360_day")

da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan])

xr.backends.netcdf3.encode_nc3_variable(xr.conventions.encode_cf_variable(da)) ```

Expected Output

Xarray can save the dataset/ an xr.Variable.

Problem Description

If there is a time variable that can be encoded using integers only, but that has a _FillValue set to NaN, saving to_netcdf(name, format="NETCDF4_CLASSIC") fails. The problem is that xarray adds a (unnecessary) _FillValue when saving a file.

Note: if the time cannot be encoded using integers only, it works: ``` python da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan]) da.encoding["units"] = "days since 2006-01-01T12:00:00"

xr.backends.netcdf3.encode_nc3_variable(xr.conventions.encode_cf_variable(da))

```

Another note: when saving with NETCDF4 ``` python da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan])

xr.backends.netCDF4_._encode_nc4_variable(xr.conventions.encode_cf_variable(da)) The following is returned: <xarray.Variable (time: 2)> array([0, 1]) Attributes: units: days since 2006-01-01 00:00:00.000000 calendar: proleptic_gregorian _FillValue: [-9223372036854775808] ```

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr da = xr.DataArray([0, 1, 2]) da.quantile(q=50)

ValueError: Percentiles must be in the range [0, 100] ```

Expected Output

python ValueError: Quantiles must be in the range [0, 1]

Problem Description

By wrapping np.nanpercentile (xref: #3559) we also get the numpy error. However, the error message is wrong as xarray needs it to be in 0..1.

BTW: thanks for #3559, makes my life easier!

Output of xr.show_versions()

Edit: uses nanpercentile internally.

MCVE Code Sample

python import xarray as xr i1 = xr.cftime_range("4991-01-01", periods=1) i2 = xr.cftime_range("7190-12-31", periods=1) i2 - i1

Expected Output

a timedelta

Problem Description

returns OverflowError: Python int too large to convert to C long. Originally I stumbled upon this when trying to open_mfdataset files from a long simulation (piControl). I did not figure out yet where this subtraction happens in open_mfdataset. (Opening the single files and using xr.concat works).

The offending lines are here

https://github.com/pydata/xarray/blob/40588dc38ddc2d573e3dc8c63b2e6533eb978656/xarray/coding/cftimeindex.py#L433

Ultimately this is probably a pandas problem as it tries to convert datetime.timedelta(days=803532) to '<m8[ns]'. pd.TimedeltaIndex has a (undocumented) dtype argument but I was not able to make anything else work (e.g. '<m8[D]').

@spencerkclark

Output of xr.show_versions()

MCVE Code Sample

```python import numpy as np import xarray as xr da_a = xr.DataArray(np.arange(3 * 2).reshape(3, 2), dims=['a', 'b']) da_b = xr.DataArray(np.arange(3 * 2 * 2).reshape(3, 2, 2), dims=['a', 'b', 'c'])

xr.dot(da_a, da_b, dims=None)

This sums over the common dimensions:python <xarray.DataArray (c: 2)> array([110, 125]) Dimensions without coordinates: c ```

To sum over all dimensions: ```python xr.dot(da_a, da_b, dims=['a', 'b', 'c'])

python <xarray.DataArray (c: 2)> array([110, 125]) Dimensions without coordinates: c ```

Problem Description

xr.dot with dims=None currently sums over all the common dimensions. However, there are cases when a sum over all dimensions is desired. E.g. xr.dot(da_a, da_b, dims=['a', 'b', 'c']) is a a memory efficient way to compute (da_a * da_b).sum() (if a and b don't share the same dimensions). This is currently used in #2922 (example).

Therefore I suggest to allow xr.dot(da_a, da_b, dims=xr.ALL_DIMS) as shortcut to sum over all dimensions.

I assume there is no intent to change the behavior of xr.dot(..., dims=None)? (As it is a bit in contrast to other functions that are mostly applied over all dimensions.)

@fujiisoup @max-sixty

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr import numpy as np

airtemps = xr.tutorial.load_dataset('air_temperature')

air = airtemps.air.isel(time=0)

levels = np.arange(225, 301, 25)

colors = ['#ffffb2', '#fecc5c', '#fd8d3c', '#e31a1c']

this does not work for me

air.plot.pcolormesh(levels=levels, colors=colors)

```

Expected Output

Should create a plot with the specified colors. According to the the docstring this should work. Or maybe I am doing something wrong here?

Problem Description

Instead I get the following error: ```python /usr/local/Miniconda3-envs/envs/2019/envs/iacpy3_2019/lib/python3.7/site-packages/xarray/plot/utils.py in _process_cmap_cbar_kwargs(func, kwargs, data) 683 # colors is only valid when levels is supplied or the plot is of type 684 # contour or contourf --> 685 if colors and (('contour' not in func.name) and (not levels)): 686 raise ValueError("Can only specify colors with contour or levels") 687

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ```

Instead I use the following, which works:

python air.plot.pcolormesh(levels=levels, cmap=colors)

Output of xr.show_versions()

Code Sample

```python import xarray as xr

a = xr.DataArray([1, 2], dims=('foo', ), name='a') b = xr.DataArray([3], dims=('bar', ), name='b')

objects = dict(a=a, b=b) xr.merge(objects)

```

Problem description

This returns an AttributeError, while the documentation states that a dict is a valid input.

``` python def merge(objects, compat='no_conflicts', join='outer'): """Merge any number of xarray objects into a single Dataset as variables.

Parameters
----------
objects : Iterable[Union[xarray.Dataset, xarray.DataArray, dict]]
    Merge together all variables from these objects. If any of them are
    DataArray objects, they must have a name.

...

```

Expected Output

Either the docs or the dict_like_objects loop should be changed.

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import xarray as xr import numpy as np

x = np.arange(100) * 365 x = xr.coding.times.decode_cf_datetime(x, 'days since 2400-01-01', calendar='proleptic_gregorian')

```

Problem description

xarray still throws an error when decoding out-of-bounds dates for proleptic_gregorian - should this be silenced?

Output of xr.show_versions()

Assume I have two datasets with the same lat/ lon grid. However, one has lon = 0...359and the other lon = -180...179. How can I wrap around one of them?

It would be nice to support nearest neighbour selection with a slice like so: ds.sel(lat=slice(30, 50), method='nearest'). Maybe the step has to be disallowed.

Or is there another possibility how to achieve this functionality?

use case: Often I look at a region with different data sets that don't have the same resolutions. It would be nice to be able to select the closest-matching region with an easy syntax.

Code Sample, a copy-pastable example if possible

```python

Your code here

import numpy as np import xarray as xr x = np.arange(10) da = xr.Dataset(data_vars=dict(data=('dim1', x)), coords=dict(dim1=('dim1', x), dim2=('dim2', x))) da.to_netcdf('tst.nc', format='NETCDF4_CLASSIC', unlimited_dims='dim1')

```

Problem description

This creates the error RuntimeError: NetCDF: NC_UNLIMITED size already in use. With format='NETCDF4' silently creates the dimensions d, i, m, and \1.

The correct syntax is unlimited_dims=['dim1'].

With format='NETCDF4_CLASSIC' and unlimited_dims=['dim1', 'dim2'], still raises the not-so-helpful NC_UNLIMITED error.

I only tested with netCDF4 as backend.

Expected Output

• better error message
• work with unlimited_dims='dim1'

Output of xr.show_versions()

Code Sample

```python

import numpy as np import xarray as xr

def test_units(units):

x = np.arange(10)
data = np.random.randn(10)

ds = xr.Dataset(data_vars=dict(data=('x', data)), coords=dict(x=('x', x)))
ds.data.attrs['units'] = units

ds.to_netcdf('tst.nc')

decoded = xr.open_dataset('tst.nc')

print(units.ljust(8), decoded.data.dtype)


ds.close()
decoded.close()

test_units('seconds') test_units('second')

test_units('minutes') test_units('minute')

test_units('days') test_units('day')

test_units('months') test_units('years') ```

Problem description

Returns: seconds timedelta64[ns] second float64 minutes timedelta64[ns] minute float64 days timedelta64[ns] day float64 months float64 years float64

Expected Output

I would expect type float for all of them. Or is this expected behaviour?

I have a dataset that reports 'consecutive dry days' and the dataset creator correctly set the units of the data to 'days', but I don't want this to be decoded (but the time axis should)....

Output of xr.show_versions()

python from xarray.plot.utils import import_seaborn _color_palette('Greys', 3) Returns 0.85, ... 0.58, ... 0.31, ...

if seaborn is installed, and

1.00, ... 0.58, ... 0.00, ...

otherwise

Problem description

the return value of _color_palette('Greys', 3) is different when seaborn is installed or not.

The relevant code is here: https://github.com/pydata/xarray/blob/6aa225f5dae9cc997e232c11a63072923c8c0238/xarray/plot/utils.py#L115 https://github.com/pydata/xarray/blob/6aa225f5dae9cc997e232c11a63072923c8c0238/xarray/plot/utils.py#L143

The same logic in seaborn https://github.com/mwaskom/seaborn/blob/0beede57152ce80ce1d4ef5d0c0f1cb61d118375/seaborn/palettes.py#L411

Intuitively I prefer the xarray solution because this uses the full range of colors which I find beneficial, however there may be a reason for this I'm not aware of.

Maybe @mwaskom will answer: mwaskom/seaborn#1372

Expected Output

Output of xr.show_versions()

~~

edit:

sorry I pressed the button to early

It would be nice if where accepts an other argument:

def where(self, cond, other=np.NaN): pass

How would I calculate percentiles over a certain dimension?

ds.groupby('time.month').apply(np.percentile, q=5)

Does not work.

Not sure if this is possible to specify in the setup.py as bottleneck is not required. However, if bottleneck is used it should probably be in version 1.0. For version 0.8.0 I get the following error:

>>> move_mean() got an unexpected keyword argument 'min_count'

Feel free to close.

If I have a netCDF file that has invalid timesteps from the 'future', it is wrongly converted to datetime64[ns].

``` import netCDF4 as nc import numpy as np import xarray as xr

create netCDF file

ncf = nc.Dataset('test_future.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1850-01-01 00:00:00' ncf.variables['time'].calendar = 'standard' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_future.nc')

this works

ds

ds.time is a datetime64[ns] object

this fails

ds.time ```

If I choose chalendar='noleap' the dates wrap around!

``` ncf = nc.Dataset('test_future_noleap.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1850-01-01 00:00:00' ncf.variables['time'].calendar = 'noleap' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_future_noleap.nc')

after 2262 they go back to 1678!

ds.time ```

If my 'invalid' time is from the 'past' it works as expected:

``` ncf = nc.Dataset('test_past.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1000-01-01 00:00:00' ncf.variables['time'].calendar = 'standard' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_past.nc')

this works

ds

ds.time is a object

ds.time ```

Sometimes it happens that you have two grids that are equal up to precision or "for practical purposes". xarray does not align them. Is there a possibility to still get them aligned (and not having to copy the coordinates from one to the other)?

Examples: - CESM output from the land model and the atmosphere model. - I have a file with a region definition on a fine grid. If I regrid that to the model resolution with cdo the grids are equal up to ~1*10**-4.

reindex can cause problems as long as the data is not loaded

``` import xarray as xr data = xr.DataArray([1, 2, 3], dims='x').to_dataset('name') data.to_netcdf('tst.nc')

works fine

a = xr.open_dataset('tst.nc') a.load() a.reindex(x=[2, 0, 1])

problem

b = xr.open_dataset('tst.nc') b.reindex(x=[2, 0, 1]) ```

When specifying a negative vmin and a positive vmax the viridis colormap is used. Is this intentional?

``` import xarray as xr import matplotlib.pyplot as plt

airtemps = xr.tutorial.load_dataset('air_temperature') air2d = airtemps.air.isel(time=500) - 273.15

diverging colormap

air2d.plot.pcolormesh() plt.show()

viridis

air2d.plot.pcolormesh(vmin=-24, vmax=24) plt.show()

------------------------------------------------------------

the solution:

diverging colormap

air2d.plot.pcolormesh(vmin=-24, center=0) plt.show() ```

EDIT: solution and working example

Now that we are at it... Why don't DataArrays have virtual variables?

It is quite convenient to select a whole year ds.sel(time='1991'). However, sel should not be used to assign data. Thus you can not do ds.sel(time='1991').coords['time'] = time and ds[dict(time='1991')] does not work.

1) Should ds[dict(time='1991')] work?

2) A possible workaround is:

idx = xray.core.indexing.remap_label_indexers(ds, {'time' : '1999'}) ds[idx] = time

Should this be added to the documentation or is this case too uncommon?

Nearest neighbor lookups does not seem to work:

import xray xray.__version__

0.4.1

data = xray.DataArray([1, 2, 3], dims='x') data.sel(x=[1.1, 1.9], method='nearest')

ValueError: not all values found in index 'x'

Example from: http://xray.readthedocs.org/en/latest/indexing.html#nearest-neighbor-lookups

(pandas version 0.16.1)

Once I take a mean over virtual variables, they are not available any more.

import pandas as pd import numpy as np import xray t = pd.date_range('2000-01-01', '2000-12-31', freq='6H') x = np.random.rand(*t.shape) time = xray.DataArray(t, name='t', dims='time') ts = xray.Dataset({'x' : ('time', x), 'time' : time}) ts_mean = ts.groupby('time.date').mean() ts_mean.virtual_variables

Is this intended behaviour? And could I get them back somehow?

Virtual variables such as time.date can not be accessed by dot notation.

``` In [81]: ts.time.date

AttributeError Traceback (most recent call last) <ipython-input-81-4206b510e310> in <module>() ----> 1 ts.time.date

/home/mathause/.local/lib/python2.7/site-packages/xray/core/common.pyc in getattr(self, name) 115 pass 116 raise AttributeError("%r object has no attribute %r" % --> 117 (type(self).name, name)) 118 119 def dir(self):

AttributeError: 'DataArray' object has no attribute 'date' ```

Compare

ts.__getattr__('time.date')

So maybe time.date is not a useful name. time_date? Or don't show these attributes to the user?