What is your issue?

I looked a bit into the frequency string update & found 3 issues we could improve upon.

1. Apart from "M", pandas also deprecated "Y", and "Q", in favor of "YE" and "QE". (And they are discussing renaming "MS" to "MB"). Should we do the same?

2. Should we translate the new freq strings to the old ones if pandas < 2.2 is installed? Otherwise we get the following situation: python import xarray as xr xr.date_range("1600-02-01", periods=3, freq="M") # deprecation warning xr.date_range("1600-02-01", periods=3, freq="ME") # ValueError: Invalid frequency: ME

3. date_range_like can emit deprecation warnings without a way to mitigate them if pandas < 2.2 is installed. (When a DatetimeIndex) is passed. Could be nice to translate the old freq string to the new one without a warning.

I have played around with 2. and 3. and can open a PR if you are on board.

@spencerkclark @aulemahal

• pandas-dev/pandas#55792
• pandas-dev/pandas#55553
• pandas-dev/pandas#56840

What happened?

As per: https://github.com/pydata/xarray/pull/8246#discussion_r1340357405

Passing a negative step in a slice to select a non-chunked zarr-backed datasets raises an error.

What did you expect to happen?

zarr should allow negative step (probably?)

Minimal Complete Verifiable Example

```Python import xarray as xr

create a zarr dataset

air = xr.tutorial.open_dataset("air_temperature") air.to_zarr("test.zarr")

ds = xr.open_dataset("test.zarr", engine="zarr") ds.air[::-1, ].load()

note that this works if the dataset is backed by dask

ds_dask = xr.open_dataset("test.zarr", engine="zarr", chunks="auto") ds_dask.air[::-1, ].load() ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python File ~/code/xarray/xarray/core/parallelcompat.py:93, in guess_chunkmanager(manager) 91 if isinstance(manager, str): 92 if manager not in chunkmanagers: ---> 93 raise ValueError( 94 f"unrecognized chunk manager {manager} - must be one of: {list(chunkmanagers)}" 95 ) 97 return chunkmanagers[manager] 98 elif isinstance(manager, ChunkManagerEntrypoint): 99 # already a valid ChunkManager so just pass through

ValueError: unrecognized chunk manager dask - must be one of: [] ```

Anything else we need to know?

The error comes from https://github.com/zarr-developers/zarr-python/blob/6ec746ef1242dd9fec26b128cc0b3455d28ad6f0/zarr/indexing.py#L174 so it would need an upstream fix first.

cc @dcherian is this what you had in mind?

Environment

What happened?

ds.weighted(weights).mean(dims) errors when reducing over a dimension that is neither on the weights nor on the variable.

What did you expect to happen?

This used to work and was "broken" by #8606. However, we may want to fix this by ignoring (?) those data vars instead (#7027).

Minimal Complete Verifiable Example

```Python import xarray as xr

ds = xr.Dataset({"a": (("y", "x"), [[1, 2]]), "scalar": 1}) weights = xr.DataArray([1, 2], dims="x")

ds.weighted(weights).mean("y") ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [ ] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python ValueError Traceback (most recent call last) Cell In[1], line 6 3 ds = xr.Dataset({"a": (("y", "x"), [[1, 2]]), "scalar": 1}) 4 weights = xr.DataArray([1, 2], dims="x") ----> 6 ds.weighted(weights).mean("y")

File ~/code/xarray/xarray/util/deprecation_helpers.py:115, in _deprecate_positional_args.<locals>._decorator.<locals>.inner(args, kwargs) 111 kwargs.update({name: arg for name, arg in zip_args}) 113 return func(args[:-n_extra_args], kwargs) --> 115 return func(*args, kwargs)

File ~/code/xarray/xarray/core/weighted.py:497, in Weighted.mean(self, dim, skipna, keep_attrs) 489 @_deprecate_positional_args("v2023.10.0") 490 def mean( 491 self, (...) 495 keep_attrs: bool | None = None, 496 ) -> T_Xarray: --> 497 return self._implementation( 498 self._weighted_mean, dim=dim, skipna=skipna, keep_attrs=keep_attrs 499 )

File ~/code/xarray/xarray/core/weighted.py:558, in DatasetWeighted._implementation(self, func, dim, kwargs) 555 def _implementation(self, func, dim, kwargs) -> Dataset: 556 self._check_dim(dim) --> 558 return self.obj.map(func, dim=dim, **kwargs)

File ~/code/xarray/xarray/core/dataset.py:6924, in Dataset.map(self, func, keep_attrs, args, kwargs) 6922 if keep_attrs is None: 6923 keep_attrs = _get_keep_attrs(default=False) -> 6924 variables = { 6925 k: maybe_wrap_array(v, func(v, *args, kwargs)) 6926 for k, v in self.data_vars.items() 6927 } 6928 if keep_attrs: 6929 for k, v in variables.items():

File ~/code/xarray/xarray/core/dataset.py:6925, in <dictcomp>(.0) 6922 if keep_attrs is None: 6923 keep_attrs = _get_keep_attrs(default=False) 6924 variables = { -> 6925 k: maybe_wrap_array(v, func(v, args, *kwargs)) 6926 for k, v in self.data_vars.items() 6927 } 6928 if keep_attrs: 6929 for k, v in variables.items():

File ~/code/xarray/xarray/core/weighted.py:286, in Weighted._weighted_mean(self, da, dim, skipna) 278 def _weighted_mean( 279 self, 280 da: T_DataArray, 281 dim: Dims = None, 282 skipna: bool | None = None, 283 ) -> T_DataArray: 284 """Reduce a DataArray by a weighted mean along some dimension(s).""" --> 286 weighted_sum = self._weighted_sum(da, dim=dim, skipna=skipna) 288 sum_of_weights = self._sum_of_weights(da, dim=dim) 290 return weighted_sum / sum_of_weights

File ~/code/xarray/xarray/core/weighted.py:276, in Weighted._weighted_sum(self, da, dim, skipna) 268 def _weighted_sum( 269 self, 270 da: T_DataArray, 271 dim: Dims = None, 272 skipna: bool | None = None, 273 ) -> T_DataArray: 274 """Reduce a DataArray by a weighted sum along some dimension(s).""" --> 276 return self._reduce(da, self.weights, dim=dim, skipna=skipna)

File ~/code/xarray/xarray/core/weighted.py:231, in Weighted._reduce(da, weights, dim, skipna) 227 da = da.fillna(0.0) 229 # dot does not broadcast arrays, so this avoids creating a large 230 # DataArray (if weights has additional dimensions) --> 231 return dot(da, weights, dim=dim)

File ~/code/xarray/xarray/util/deprecation_helpers.py:140, in deprecate_dims.<locals>.wrapper(args, kwargs) 132 emit_user_level_warning( 133 "The dims argument has been renamed to dim, and will be removed " 134 "in the future. This renaming is taking place throughout xarray over the " (...) 137 PendingDeprecationWarning, 138 ) 139 kwargs["dim"] = kwargs.pop("dims") --> 140 return func(args, **kwargs)

File ~/code/xarray/xarray/core/computation.py:1885, in dot(dim, arrays, *kwargs) 1883 dim = tuple(d for d, c in dim_counts.items() if c > 1) 1884 else: -> 1885 dim = parse_dims(dim, all_dims=tuple(all_dims)) 1887 dot_dims: set[Hashable] = set(dim) 1889 # dimensions to be parallelized

File ~/code/xarray/xarray/core/utils.py:1046, in parse_dims(dim, all_dims, check_exists, replace_none) 1044 dim = (dim,) 1045 if check_exists: -> 1046 _check_dims(set(dim), set(all_dims)) 1047 return tuple(dim)

File ~/code/xarray/xarray/core/utils.py:1131, in _check_dims(dim, all_dims) 1129 if wrong_dims: 1130 wrong_dims_str = ", ".join(f"'{d!s}'" for d in wrong_dims) -> 1131 raise ValueError( 1132 f"Dimension(s) {wrong_dims_str} do not exist. Expected one or more of {all_dims}" 1133 )

ValueError: Dimension(s) 'y' do not exist. Expected one or more of {'x'} ```

Anything else we need to know?

No response

Environment

Newest main (i.e. 2024.01)

What happened?

The html repr of a cftime array wants to display the "calendar", which it cannot if it is empty.

What did you expect to happen?

No error.

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

data_obs = np.random.randn(3) time_obs = xr.date_range("2000-01-01", periods=3, freq="YS", calendar="noleap")

obs = xr.DataArray(data_obs, coords={"time": time_obs})

o = obs[:0]

xr.core.formatting_html.array_repr(o) ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python ValueError Traceback (most recent call last) Input In [1], in <cell line: 12>() 8 obs = xr.DataArray(data_obs, coords={"time": time_obs}) 10 o = obs[:0] ---> 12 xr.core.formatting_html.array_repr(o)

File ~/code/xarray/xarray/core/formatting_html.py:318, in array_repr(arr) 316 if hasattr(arr, "xindexes"): 317 indexes = _get_indexes_dict(arr.xindexes) --> 318 sections.append(index_section(indexes)) 320 sections.append(attr_section(arr.attrs)) 322 return _obj_repr(arr, header_components, sections)

File ~/code/xarray/xarray/core/formatting_html.py:195, in _mapping_section(mapping, name, details_func, max_items_collapse, expand_option_name, enabled) 188 expanded = _get_boolean_with_default( 189 expand_option_name, n_items < max_items_collapse 190 ) 191 collapsed = not expanded 193 return collapsible_section( 194 name, --> 195 details=details_func(mapping), 196 n_items=n_items, 197 enabled=enabled, 198 collapsed=collapsed, 199 )

File ~/code/xarray/xarray/core/formatting_html.py:155, in summarize_indexes(indexes) 154 def summarize_indexes(indexes): --> 155 indexes_li = "".join( 156 f"

{indexes_li}

File ~/code/xarray/xarray/core/formatting_html.py:156, in <genexpr>(.0) 154 def summarize_indexes(indexes): 155 indexes_li = "".join( --> 156 f"

{indexes_li}

File ~/code/xarray/xarray/core/formatting_html.py:140, in summarize_index(coord_names, index) 138 index_id = f"index-{uuid.uuid4()}" 139 preview = escape(inline_index_repr(index)) --> 140 details = short_index_repr_html(index) 142 data_icon = _icon("icon-database") 144 return ( 145 f"

File ~/code/xarray/xarray/core/formatting_html.py:132, in short_index_repr_html(index) 129 if hasattr(index, "repr_html"): 130 return index.repr_html() --> 132 return f"

{escape(repr(index))}

File ~/code/xarray/xarray/core/indexes.py:547, in PandasIndex.repr(self) 546 def repr(self): --> 547 return f"PandasIndex({repr(self.index)})"

File ~/code/xarray/xarray/coding/cftimeindex.py:353, in CFTimeIndex.repr(self) 345 end_str = format_times( 346 self.values[-REPR_ELLIPSIS_SHOW_ITEMS_FRONT_END:], 347 display_width, 348 offset=offset, 349 first_row_offset=offset, 350 ) 351 datastr = "\n".join([front_str, f"{' '*offset}...", end_str]) --> 353 attrs_str = format_attrs(self) 354 # oneliner only if smaller than display_width 355 full_repr_str = f"{klass_name}([{datastr}], {attrs_str})"

File ~/code/xarray/xarray/coding/cftimeindex.py:272, in format_attrs(index, separator) 267 def format_attrs(index, separator=", "): 268 """Format attributes of CFTimeIndex for repr.""" 269 attrs = { 270 "dtype": f"'{index.dtype}'", 271 "length": f"{len(index)}", --> 272 "calendar": f"'{index.calendar}'", 273 "freq": f"'{index.freq}'" if len(index) >= 3 else None, 274 } 276 attrs_str = [f"{k}={v}" for k, v in attrs.items()] 277 attrs_str = f"{separator}".join(attrs_str)

File ~/code/xarray/xarray/coding/cftimeindex.py:698, in CFTimeIndex.calendar(self) 695 """The calendar used by the datetimes in the index.""" 696 from .times import infer_calendar_name --> 698 return infer_calendar_name(self)

File ~/code/xarray/xarray/coding/times.py:374, in infer_calendar_name(dates) 371 return sample.calendar 373 # Error raise if dtype is neither datetime or "O", if cftime is not importable, and if element of 'O' dtype is not cftime. --> 374 raise ValueError("Array does not contain datetime objects.")

ValueError: Array does not contain datetime objects. ```

Anything else we need to know?

Bisected to 7379923de756a2bcc59044d548f8ab7a68b91d4e use _repr_inline_ for indexes that define it.

Environment

What happened?

da.xindexes or da.indexes raises an error when trying to generate the repr if there are no coords (indexes)

What did you expect to happen?

Displaying an empty Mappable?

Minimal Complete Verifiable Example

Python xr.DataArray([3, 5]).indexes xr.DataArray([3, 5]).xindexes

MVCE confirmation

• [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [x] Complete example — the example is self-contained, including all data and the text of any traceback.
• [x] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [x] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python Out[9]: --------------------------------------------------------------------------- ValueError Traceback (most recent call last) File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/core/formatters.py:708, in PlainTextFormatter.call(self, obj) 701 stream = StringIO() 702 printer = pretty.RepresentationPrinter(stream, self.verbose, 703 self.max_width, self.newline, 704 max_seq_length=self.max_seq_length, 705 singleton_pprinters=self.singleton_printers, 706 type_pprinters=self.type_printers, 707 deferred_pprinters=self.deferred_printers) --> 708 printer.pretty(obj) 709 printer.flush() 710 return stream.getvalue()

File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/lib/pretty.py:410, in RepresentationPrinter.pretty(self, obj) 407 return meth(obj, self, cycle) 408 if cls is not object \ 409 and callable(cls.dict.get('repr')): --> 410 return _repr_pprint(obj, self, cycle) 412 return _default_pprint(obj, self, cycle) 413 finally:

File ~/.conda/envs/xarray_dev/lib/python3.10/site-packages/IPython/lib/pretty.py:778, in repr_pprint(obj, p, cycle) 776 """A pprint that just redirects to the normal repr function.""" 777 # Find newlines and replace them with p.break() --> 778 output = repr(obj) 779 lines = output.splitlines() 780 with p.group():

File ~/code/xarray/xarray/core/indexes.py:1659, in Indexes.repr(self) 1657 def repr(self): 1658 indexes = formatting._get_indexes_dict(self) -> 1659 return formatting.indexes_repr(indexes)

File ~/code/xarray/xarray/core/formatting.py:474, in indexes_repr(indexes, max_rows) 473 def indexes_repr(indexes, max_rows: int | None = None) -> str: --> 474 col_width = _calculate_col_width(chain.from_iterable(indexes)) 476 return _mapping_repr( 477 indexes, 478 "Indexes", (...) 482 max_rows=max_rows, 483 )

File ~/code/xarray/xarray/core/formatting.py:341, in _calculate_col_width(col_items) 340 def _calculate_col_width(col_items): --> 341 max_name_length = max(len(str(s)) for s in col_items) if col_items else 0 342 col_width = max(max_name_length, 7) + 6 343 return col_width

ValueError: max() arg is an empty sequence ```

Anything else we need to know?

No response

Environment

I really should not open this can of worms but per https://github.com/pydata/xarray/issues/4450#issuecomment-697507489:

I'm always confused about whether ds.groupby(..., keep_attrs=True).mean() or ds.groupby(...).mean(keep_attrs=True) is correct. (similarly for rolling, coarsen etc.)

Also as I try to fix the keep_attr behavior in #4510 it would be good to know where they should go. So I tried to figure out how this is currently handled and found the following:

ds.xxx(keep_attrs=True).yyy() - all fixed

ds.xxx().yyy(keep_attrs=True) - coarsen (fixed in #5227) - groupby - groupby_bin - resample - rolling (adjusted in #4510) - rolling_exp (fixed in #4592) - weighted

So the working consensus seems to be to to ds.xxx().yyy(keep_attrs=True) - any comments on that?

(Edit: looking at this it is only half as bad, "only" coarsen, rolling (#4510), and rolling_exp would need to be fixed.)

Detailed analysis

edit: moved rolling after #4510, moved rolling_exp after #4592 and coarsen after #5227

What happened?

I tried a DataArray with more than one dimension coordinate. Unfortunately xr.align fails, which disallows any arithmetic operation - even when the coords are exactly the same.

What did you expect to happen?

No response

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

data = np.arange(12).reshape(3, 4)

y = [10, 20, 30] s = ["a", "b", "c"]

x = [1, 2, 3, 4]

da = xr.DataArray(data, dims=("y", "x"), coords={"x": x, "y": y, "s": ("y", s)}) da = da.set_xindex("s")

xr.align(da, da.y) # errors da + da # errors da + da.x # errors ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [ ] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

```Python

ValueError Traceback (most recent call last) /home/mathause/code/mesmer/devel/prepare_for_surfer.ipynb Cell 28 line 1 12 da = xr.DataArray(data, dims=("y", "x"), coords={"x": x, "y": y, "s": ("y", s)}) 13 da = da.set_xindex("s") ---> 15 xr.align(da, da.y) # errors 17 da + da.x # errors

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:888, in align(join, copy, indexes, exclude, fill_value, *objects) 692 """ 693 Given any number of Dataset and/or DataArray objects, returns new 694 objects with aligned indexes and dimension sizes. ref='~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:0'>0;32m (...) 878 879 """ 880 aligner = Aligner( 881 objects, 882 join=join, ref='~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:0'>0;32m (...) 886 fill_value=fill_value, 887 ) --> 888 aligner.align() 889 return aligner.results

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:573, in Aligner.align(self) 571 self.find_matching_indexes() 572 self.find_matching_unindexed_dims() --> 573 self.assert_no_index_conflict() 574 self.align_indexes() 575 self.assert_unindexed_dim_sizes_equal()

File ~/.conda/envs/mesmer_dev/lib/python3.9/site-packages/xarray/core/alignment.py:318, in Aligner.assert_no_index_conflict(self) 314 if dup: 315 items_msg = ", ".join( 316 f"{k!r} ({v} conflicting indexes)" for k, v in dup.items() 317 ) --> 318 raise ValueError( 319 "cannot re-index or align objects with conflicting indexes found for " 320 f"the following {msg}: {items_msg}\n" 321 "Conflicting indexes may occur when\n" 322 "- they relate to different sets of coordinate and/or dimension names\n" 323 "- they don't have the same type\n" 324 "- they may be used to reindex data along common dimensions" 325 )

ValueError: cannot re-index or align objects with conflicting indexes found for the following dimensions: 'y' (2 conflicting indexes) Conflicting indexes may occur when - they relate to different sets of coordinate and/or dimension names - they don't have the same type - they may be used to reindex data along common dimensions ```

Anything else we need to know?

No response

Environment

What happened?

Running an in-memory groupby operation took much longer than expected. Turning off flox fixed this - but I don't think that's the idea ;-)

What did you expect to happen?

flox to be at least on par with our naive implementation

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr

arr = np.random.randn(10, 10, 36530) time = xr.date_range("2000", periods=30365, calendar="noleap") da = xr.DataArray(arr, dims=("y", "x", "time"), coords={"time": time})

using max

print("max:") xr.set_options(use_flox=True) %timeit da.groupby("time.year").max("time") %timeit da.groupby("time.year").max("time", engine="flox")

xr.set_options(use_flox=False) %timeit da.groupby("time.year").max("time")

as reference

%timeit [da.sel(time=str(year)).max("time") for year in range(2000, 2030)]

using mean

print("mean:") xr.set_options(use_flox=True) %timeit da.groupby("time.year").mean("time") %timeit da.groupby("time.year").mean("time", engine="flox")

xr.set_options(use_flox=False) %timeit da.groupby("time.year").mean("time")

as reference

%timeit [da.sel(time=str(year)).mean("time") for year in range(2000, 2030)] ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python max: 158 ms ± 4.41 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 28.1 ms ± 318 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) 11.5 ms ± 52.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

mean: 95.6 ms ± 10.8 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) 34.8 ms ± 2.88 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) 15.2 ms ± 232 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) ```

Anything else we need to know?

No response

Environment

In #4592 I had the issue that mypy did not raise an error on a missing method:

```python from xarray.core.common import DataWithCoords

hasattr(xr.core.common.DataWithCoords, "reduce") # -> False

def test(x: "DataWithCoords"): x.reduce() # mypy does not error ```

This is because DataWithCoords implements __getattr__:

```python

class A: pass

class B: def getattr(self, name): ...

def testA(x: "A"): x.reduce() # mypy errors

def testB(x: "B"): x.reduce() # mypy does not error ```

The solution seems to be to not typecheck __getattr__ (see https://github.com/python/mypy/issues/6251#issuecomment-457287161):

```python from typing import no_type_check

class C: @no_type_check def getattr(self, name): ...

def testC(x: "C"): x.reduce() # mypy errors ```

The only __getattr__ within xarray is here:

https://github.com/pydata/xarray/blob/17358922d480c038e66430735bf4c365a7677df8/xarray/core/common.py#L221

Using @no_type_check leads to 24 errors and not all of them can be trivially solved. E.g. DataWithCoords wants of use self.isel but does not implement the method. The solution is probably to add isel to DataWithCoords as an ABC or using NotImplemented.

Thoughts?

All errors

Edit: one problem is certainly the method injection, as mypy cannot detect those types.

When I manually specify the units attribute for time, and then save the Dataset to_netcdf the string is truncated. See exaple

import pandas as pd
import xarray as xr

time = pd.date_range('2000-01-01', '2000-01-31', freq='6h')
ds = xr.Dataset(coords=dict(time=time))

units = 'days since 1975-01-01 00:00:00'
calendar = 'gregorian'
encoding=dict(time=dict(units=units, calendar=calendar))

ds.to_netcdf('test.nc', format='NETCDF4_CLASSIC', encoding=encoding)

! ncdump -h test.nc
# time:units = "days since 1975-01-01" ;

Some programs seem to require the hours to be present to interpret the time properly (e.g. panoply). When specifying the hour, a 'T' is added.

units = 'days since 1975-01-01 01:00:00'

! ncdump -h test.nc
# time:units = "days since 1975-01-01T01:00:00" ;

When xarray defines the time.units it works fine.

ds = xr.Dataset(coords=dict(time=time))
ds.to_netcdf('test.nc', format='NETCDF4_CLASSIC',)

! ncdump -h test.nc
# time:units = "hours since 2000-01-01 00:00:00" ;

xarray version 0.9.6

What is your issue?

Applying e.g. ds.weighted(weights).mean() applies the operation over all DataArray objects - even if they don't have the dimensions over which it is applied (or is a scalar variable). I don't think this is wanted.

```python import xarray as xr

air = xr.tutorial.open_dataset("air_temperature") air.attrs = {}

add variable without dims

air["foo"] = 5

print("resample") print(air.resample(time="MS").mean(dim="time").foo.dims)

print("groupby") print(air.groupby("time.year").mean(dim="time").foo.dims)

print("weighted") print(air.weighted(weights=air.time.dt.year).mean("lat").foo.dims)

print("where") print(air.where(air.air > 5).foo.dims) ```

Results resample ('time',) groupby ('year',) weighted ('time',)

Related #6952 - I am sure there are other issues, but couldn't find them quickly...

rolling and coarsen don't seem to do this.

What happened?

A cftime variable returns a timedelta64[ns] when calling diff / + / - and it can then not be added/ subtracted from the original data.

What did you expect to happen?

We can add cftime timedeltas.

Minimal Complete Verifiable Example

```Python import xarray as xr

air = xr.tutorial.open_dataset("air_temperature", use_cftime=True)

air.time + air.time.diff("time") / 2 ```

MVCE confirmation

• [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [x] Complete example — the example is self-contained, including all data and the text of any traceback.
• [x] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Python air.time.variable.values[1:] - air.time.variable.values[:-1] returns array([datetime.timedelta(seconds=21600), ...]) but then Python xr.Variable(("time",), np.array([datetime.timedelta(0)])) returns a dtype='timedelta64[ns]' array.

Anything else we need to know?

• See upstream PR: xarray-contrib/cf-xarray#441
• Similar to #7381 (but I don't think it's the same issue, feel free to close if you disagree)
• That might need a special data type for timedeltas of cftime.Datetime objects, or allowing to add 'timedelta64[ns]' to cftime.Datetime objects
• The casting comes from

https://github.com/pydata/xarray/blob/d8ec3a3f6b02a8b941b484b3d254537af84b5fde/xarray/core/variable.py#L366

https://github.com/pydata/xarray/blob/d8ec3a3f6b02a8b941b484b3d254537af84b5fde/xarray/core/variable.py#L272

Environment

It would be nice to make weighted work with groupby - e.g. #3935 (comment)

However, it is not entirely clear to me how that should be done. One way would be to do: python da.groupby(...).weighted(weights).mean() this would require that the groupby operation is applied over the weights (how would this be done?) Or should it be

python da.weighted(weights).groupby(...).mean() but this seems less intuitive to me.

Or python da.groupby(..., weights=weights).mean()

We are currently not testing with the newest version of netCDF4 because it is incompatible with pynio (the newest version is 1.5.4, we are at 1.5.3). This is unlikely to be fixed, see conda-forge/pynio-feedstock#90.

Therefore we need to think how to setup the tests so we use the newest version of netCDF4. Maybe just remove it from py38.yml?

And long term what to do with the pynio backend? Deprecate? Move to an external repo?

What happened?

xr.core.dataset.DataVariables assumes that everything that is in ds._dataset._variables and not in self._dataset._coord_names is a "data variable". However, since the index refactor we can end up with more _coord_names than _variables which breaks a number of stuff (e.g. the repr).

What did you expect to happen?

Well it seems this assumption is now wrong.

Minimal Complete Verifiable Example

Python ds = xr.Dataset(coords={"a": ("x", [1, 2, 3]), "b": ("x", ['a', 'b', 'c'])}) ds.set_index(z=['a', 'b']).reset_index("z", drop=True)

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Python ValueError: __len__() should return >= 0

Anything else we need to know?

The error comes from here

https://github.com/pydata/xarray/blob/63ba862d03c8d0cd8b44d2071bc360e9fed4519d/xarray/core/dataset.py#L368

Bisected to #5692 - which probably does not help too much.

Environment

What happened?

We generally type dimensions as:

python dims: Hashable | Iterable[Hashable]

However, this is in conflict with passing a tuple of independent dimensions to a method - e.g. da.mean(("x", "y")) because a tuple is also hashable.

Also mypy requires an isinstance(dims, Hashable) check when typing a function. We use an isinstance(dims, str) check in many places to wrap a single dimension in a list. Changing this to isinstance(dims, Hashable) will change the behavior for tuples.

What did you expect to happen?

In the community call today we discussed to change this to

python dims: str | Iterable[Hashable]

i.e. if a single dim is passed it has to be a string and wrapping it in a list is a convenience function. Special use cases with Hashable types should be wrapped in a Iterable by the user. This probably best reflects the current state of the repo (dims = [dims] if isinstance(dims, str) else dims).

The disadvantage could be that it is a bit more difficult to explain in the docstrings?

@shoyer - did I get this right from the discussion?

Other options

1. Require str as dimension names.

This could be too restrictive. @keewis mentioned that tuple dimension names are already used somwehere in the xarray repo. Also we discussed in another issue or PR (which I cannot find right know) that we want to keep allowing Hashable.

1. Disallow passing tuples (only allow tuples if a dimension is a tuple), require lists to pass several dimensions.

This is too restrictive in the other direction and will probably lead to a lot of downstream troubles. Naming a single dimension with a tuple will be a very rare case, in contrast to passing several dimension names as a tuple.

1. Special case tuples. We could potentially check if dims is a tuple and if there are any dimension names consisting of a tuple. Seems more complicated and potentially brittle for probably small gains (IMO).

Minimal Complete Verifiable Example

No response

Relevant log output

No response

Anything else we need to know?

• We need to check carefully where general Hashable are really allowed. E.g. dims of a DataArray are typed as

https://github.com/pydata/xarray/blob/e056cacdca55cc9d9118c830ca622ea965ebcdef/xarray/core/dataarray.py#L380

but tuples are not actually allowed:

```python import xarray as xr xr.DataArray([1], dims=("x", "y"))

ValueError: different number of dimensions on data and dims: 1 vs 2

xr.DataArray([1], dims=[("x", "y")])

TypeError: dimension ('x', 'y') is not a string

```

• We need to be careful typing functions where only one dim is allowed, e.g. xr.concat, which should probably set dim: Hashable (and make sure it works).
• Do you have examples for other real-world hashable types except for str and tuple? (Would be good for testing purposes).

Environment

N/A

polyfit currently only allows to fit along a dimension and not along a non-dimension coordinate (or a virtual coordinate)

Example: ```python da = xr.DataArray( [1, 3, 2], dims=["x"], coords=dict(x=["a", "b", "c"], y=("x", [0, 1, 2])) )

print(da)

da.polyfit("y", 1) Output:python <xarray.DataArray (x: 3)> array([1, 3, 2]) Coordinates: * x (x) <U1 'a' 'b' 'c' y (x) int64 0 1 2

KeyError Traceback (most recent call last) <ipython-input-80-9bb2dacf50f7> in <module> 5 print(da) 6 ----> 7 da.polyfit("y", 1)

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/dataarray.py in polyfit(self, dim, deg, skipna, rcond, w, full, cov) 3507 """ 3508 return self._to_temp_dataset().polyfit( -> 3509 dim, deg, skipna=skipna, rcond=rcond, w=w, full=full, cov=cov 3510 ) 3511

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/dataset.py in polyfit(self, dim, deg, skipna, rcond, w, full, cov) 6005 skipna_da = skipna 6006 -> 6007 x = get_clean_interp_index(self, dim, strict=False) 6008 xname = "{}_".format(self[dim].name) 6009 order = int(deg) + 1

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/missing.py in get_clean_interp_index(arr, dim, use_coordinate, strict) 246 247 if use_coordinate is True: --> 248 index = arr.get_index(dim) 249 250 else: # string

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/common.py in get_index(self, key) 378 """ 379 if key not in self.dims: --> 380 raise KeyError(key) 381 382 try:

KeyError: 'y' ```

Describe the solution you'd like

Would be nice if that worked.

Describe alternatives you've considered

One could just set the non-dimension coordinate as index, e.g.: da = da.set_index(x="y")

Additional context

Allowing this may be as easy as replacing

https://github.com/pydata/xarray/blob/9c85dd5f792805bea319f01f08ee51b83bde0f3b/xarray/core/missing.py#L248

by index = arr[dim] but I might be missing something. Or probably a use_coordinate must be threaded through to get_clean_interp_index (although I am a bit confused by this argument).

This started as a question and I add it as reference. Maybe you have a comment.

There are several ways to calculate time series of seasonal data (starting from monthly or daily data):

```

load libraries

import pandas as pd import matplotlib.pyplot import numpy as np import xarray as xr

Create Example Dataset

time = pd.date_range('2000.01.01', '2010.12.31', freq='M') data = np.random.rand(*time.shape) ds = xr.DataArray(data, coords=dict(time=time))

(1) using resample

ds_res = ds.resample('Q-FEB', 'time') ds_res = ds_res.sel(time=ds_res['time.month'] == 2) ds_res = ds_res.groupby('time.year').mean('time')

(2) this is wrong

ds_season = ds.where(ds['time.season'] == 'DJF').groupby('time.year').mean('time')

(3) using where and rolling

mask other months with nan

ds_DJF = ds.where(ds['time.season'] == 'DJF')

rolling mean -> only Jan is not nan

however, we loose Jan/ Feb in the first year and Dec in the last

ds_DJF = ds_DJF.rolling(min_periods=3, center=True, time=3).mean()

make annual mean

ds_DJF = ds_DJF.groupby('time.year').mean('time')

ds_res.plot(marker='*') ds_season.plot() ds_DJF.plot()

plt.show() ```

(1) The first is to use resample with 'Q-FEB' as argument. This works fine. It does include Jan/ Feb in the first year, and Dec in the last year + 1. If this makes sense can be debated. One case where this does not work is when you have, say, two regions in your data set, for one you want to calculate DJF and for the other you want NovDecJan.

(2) Using 'time.season' is wrong as it combines Jan, Feb and Dec from the same year.

(3) The third uses where and rolling and you lose 'incomplete' seasons. If you replace ds.where(ds['time.season'] == 'DJF') with ds.groupby('time.month').where(summer_months), where summer_months is a boolean array it works also for non-standard 'summers' (or seasons) across the globe.

Code Sample, a copy-pastable example if possible

```python

Your code here

import xarray as xr import numpy as np x = np.arange(10.) da = xr.Dataset(data_vars=dict(data=('dim1', x)), coords=dict(dim1=('dim1', x))) da.to_netcdf('tst.nc')

```

Problem description

Apologies if this was discussed somwhere and it probably does not matter much, but tst.nc has _FillValue although it is not really necessary.

Output of xr.show_versions()

Is your feature request related to a problem?

xr.broadcast always does an outer join:

https://github.com/pydata/xarray/blob/de965f342e1c9c5de92ab135fbc4062e21e72453/xarray/core/alignment.py#L702

https://github.com/pydata/xarray/blob/de965f342e1c9c5de92ab135fbc4062e21e72453/xarray/core/alignment.py#L768

This is not how the (default) broadcasting (arithmetic join) works, e.g. the following first does an inner join and then broadcasts:

```python import xarray as xr

da1 = xr.DataArray([[0, 1, 2]], dims=("y", "x"), coords={"x": [0, 1, 2]}) da2 = xr.DataArray([0, 1, 2, 3, 4], dims="x", coords={"x": [0, 1, 2, 3, 4]}) da1 + da2 ```

<xarray.DataArray (y: 1, x: 3)> array([[0, 2, 4]]) Coordinates: * x (x) int64 0 1 2 Dimensions without coordinates: y

Describe the solution you'd like

Add a join argument to xr.broadcast. I would propose to leave the default as is

python def broadcast(*args, exclude=None, join="outer"): args = align(*args, join=join, copy=False, exclude=exclude)

Describe alternatives you've considered

• We could make broadcast respect options -> arithmetic_join but that would be a breaking change and I am not sure how the deprecation should/ would be handled...
• We could leave it as is.

Additional context

• xr.broadcast should not be used often because this is should happen automatically in most cases
• in #6059 I use broadcast because I couldn't get it to work otherwise (maybe there is a better way?). However, the "outer elements" are immediately discarded again - so it's kind of pointless to do an outer join.

```python import numpy as np import xarray as xr

da = xr.DataArray(np.arange(6).reshape(3, 2), coords={"dim_0": [0, 1, 2]}) w = xr.DataArray([1, 1, 1, 1, 1, 1], coords={"dim_0": [0, 1, 2, 4, 5, 6]}) da.weighted(w).quantile(0.5) ```

bottleneck cannot be built with python 3.10. See https://github.com/pydata/xarray/actions/runs/1731371015

pandas deprecates the method keyword in Index.get_loc, see pandas-dev/pandas#42269. Therefore we end up with about 5000 warnings in our upstream tests:

FutureWarning: Passing method to Index.get_loc is deprecated and will raise in a future version. Use index.get_indexer([item], method=...) instead

We should fix this before pandas releases because the warning will not be silent (FutureWarning) or ask pandas to give us more time and use a DeprecationWarning at the moment.

@benbovy @spencerkclark

```python import xarray as xr import numpy as np

ds = xr.DataArray([np.NaN, np.NaN])

ds.mean() ds.sum()

```

Problem description

ds.mean() returns NaN, ds.sum() returns 0. This comes from numpy (cp np.nanmean vs. np.nansum), so it might have to be discussed upstream, but I wanted to ask the xarray community first on their opinion. This is also relevant for #422 (what happens if the all weights are NaN or sum up to 0).

Expected Output

I would expect both to return np.nan.

Output of xr.show_versions()

I disabled pytest-xdist because a dask issue renders our CI unusable. As soon as dask releases a new version we should revert #6077 again.

Code Sample, a copy-pastable example if possible

```python import numpy as np import xarray as xr

x = np.arange(1, 366) y = np.random.randn(365) ds = xr.DataArray(y, dims=dict(dayofyear=x))

ds.rolling(center=True, dayofyear=31).mean() ```

Problem description

rolling cannot directly handle periodic boundary conditions (lon, dayofyear, ...), but could be very helpful to e.g. calculate climate indices. Also I cannot really think of an easy way to append the first elements to the end of the dataset and then calculate rolling.

Is there a way to do this? Should xarray support this feature?

This might also belong to SO...

What happened:

combine_by_coords can succeed when it should not - depending on the name of the dimensions (which determines the order of operations in combine_by_coords).

What you expected to happen:

• I think it should throw an error in both cases.

Minimal Complete Verifiable Example:

```python import numpy as np import xarray as xr

data = np.arange(5).reshape(1, 5) x = np.arange(5) x_name = "lat"

da0 = xr.DataArray(data, dims=("t", x_name), coords={"t": [1], x_name: x}).to_dataset(name="a") x = x + 1e-6 da1 = xr.DataArray(data, dims=("t", x_name), coords={"t": [2], x_name: x}).to_dataset(name="a") ds = xr.combine_by_coords((da0, da1))

ds ```

returns: python <xarray.Dataset> Dimensions: (lat: 10, t: 2) Coordinates: * lat (lat) float64 0.0 1e-06 1.0 1.0 2.0 2.0 3.0 3.0 4.0 4.0 * t (t) int64 1 2 Data variables: a (t, lat) float64 0.0 nan 1.0 nan 2.0 nan ... 2.0 nan 3.0 nan 4.0 Thus lat is interlaced - it don't think combine_by_coords should do this. If you set

python x_name = "lat" and run the example again, it returns:

```python-traceback ValueError: Resulting object does not have monotonic global indexes along dimension x

```

Anything else we need to know?:

• this is vaguely related to #4077 but I think it is separate
• combine_by_coords concatenates over all dimensions where the coords are different - therefore compat="override" doesn't actually do anything? Or does it?

https://github.com/pydata/xarray/blob/ba42c08af9afbd9e79d47bda404bf4a92a7314a0/xarray/core/combine.py#L69

cc @dcherian @TomNicholas

Environment:

In #4622 @toddrjen points out that xr.testing.assert_equal does not test for the dtype, only for the value. Therefore the following does not raise an error:

```python import numpy as np import xarray as xr import pandas as pd

xr.testing.assert_equal( xr.DataArray(np.array(1, dtype=int)), xr.DataArray(np.array(1, dtype=float)) ) xr.testing.assert_equal( xr.DataArray(np.array(1, dtype=int)), xr.DataArray(np.array(1, dtype=object)) ) xr.testing.assert_equal( xr.DataArray(np.array("a", dtype=str)), xr.DataArray(np.array("a", dtype=object)) ) ```

This comes back to numpy, i.e. the following is True:

python np.array(1, dtype=int) == np.array(1, dtype=float)

Depending on the situation one or the other is desirable or not. Thus, I would suggest to add a check_dtype argument to xr.testing.assert_equal and also to DataArray.equals (and Dataset and Variable and identical). I have not seen such an option in numpy, but pandas has it (e.g. pd.testing.assert_series_equal(left, right, check_dtype=True, ...). I would not change __eq__.

• Thoughts?
• What should the default be? We could try True first and see how many failures this creates?
• What to do with coords and indexes? pd.testing.assert_series_equal has a check_index_type keyword. Probably we need check_coords_type as well? This makes the whole thing much more complicated... Also #4543

MCVE Code Sample

```python import numpy as np import xarray as xr import matplotlib as mpl import matplotlib.pyplot as plt

data = xr.DataArray(np.arange(24).reshape(4, 6))

data.plot.contour(vmax=10, add_colorbar=True) ```

Expected Output

python h = plt.contour(data.values, vmax=10) plt.colorbar(h)

Problem Description

A contour(vmax=vmax) plot differs between xarray and matplotlib. I think the problem is here:

https://github.com/pydata/xarray/blob/95e4f6c7a636878c94b892ee8d49866823d0748f/xarray/plot/utils.py#L265

xarray calculates the levels from vmax while matplotlib (probably) calculates the levels from data.max() and uses vmax only for the norm. For contourf and pcolormesh this is not so relevant as the capped values are then drawn with the over color. However, there may also be a good reason for this behavior.

Output of xr.show_versions()

There was a discussion on the performance of the weighted mean/ sum in terms of memory footprint but also speed, and there may indeed be some things that can be optimized. See the posts at the end of the PR. However, the optimal implementation will probably depend on the use case and some profiling will be required.

I'll just open an issue to keep track of this. @seth-p

What happened:

I could not open the test dataset on master. It's a bit strange that this is not picked up by the tests, so probably something to do with the environment I have (I just updated all packages).

@alexamici @aurghs does that tell you anything?

I can also try to figure it out.

Minimal Complete Verifiable Example:

calling open_dataset with "" is enough to trigger the error:

python import xarray as xr air = xr.open_dataset("")

Anything else we need to know?:

And the traceback:

Environment:

and my conda list:

edit: added the traceback

Today I stumbled over a small pitfall, which I think could be avoided:

I am working with arrays that have axes labeled with categorical values and I ended up using True/False as labels for some binary categories:

python test = xarray.DataArray( numpy.ones((3,2)), dims=["binary","ternary"], coords={"ternary":[3,7,9],"binary":[False,True]} ) now came the big surprise, when I wanted to reduce over selections of the data:

test.sel(ternary=[9,3,7]) # does exactly what I expect and gives me the correctly permuted 3x2 array test.sel(binary=[True,False]) # does not do what I expect Instead of using the coordinate values like with the ternary category, it uses the list as boolean mask and hence I get a 3x1 array at the binary=False coordinate.

I assume that this behavior is reasonable in most cases - And I for sure will stop using bools as binary category labels. That said in the above case the conceptually identical call results in completely different outcome.

My (radical) proposal would be: forbid binary coordinates in general to avoid such confusion.

Curious about your thoughts! Hth,

Marti

Originally posted by @martinitus in https://github.com/pydata/xarray/discussions/4861

Looking at some warnings in #3266 I saw that matplotlib will deprecate the old behaviour of pcolormesh when the shape of the data and the coordinates are equal (they silently cut a row and a column of the data). With the new behaviour they will interpolate the coordinates.

```python import numpy as np import matplotlib.pyplot as plt

x = np.array([1, 2, 3]) y = np.array([1, 2, 3, 4, 5])

data = np.random.randn(*y.shape + x.shape)

f, axes = plt.subplots(1, 2)

for ax, shading, behavior in zip(axes, ["flat", "nearest"], ["old", "new"]): ax.pcolormesh(x, y, data, shading=shading, vmin=-0.75, vmax=0.75) ax.set_title(f"{behavior}: shading='{shading}'") ```

This is a good thing in general - we already do this for a long time with the infer_intervals keyword. Unfortunately they don't check if the data is monotonic (matplotlib/matplotlib#18317) which can lead to problems for maps (scitools/cartopy#1638). I don't think there is a need to do something right now - let's see what they think upstream.

This change was introduced in mpl 3.3.0

We currently pin netCDF4 to version 1.5. However, I think netCDF4 does not really follow semantic versioning, e.g. python 2 support was dropped in version 1.5.6. So they may actually be doing something like 1.major.minor[.patch] - I asked about their versioning scheme in Unidata/netcdf4-python#1090.

So I wonder if we would need to pin netCDF to version to version 1.5.4.

The last 5 days our Upstream CI failed silently with a timeout after 6h:

https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml?query=branch%3Amaster+event%3Aschedule

This was probably caused by #4934. As mentioned in dask/dask#4934 this is probably dask/dask#6738 which was merged 5 days ago.

We currently run mypy in the pre-commit hooks CI. However, this is done in an environment where no dependencies are installed. Therefore we missed the errors that pop up when running mypy with numpy 1.20 installed. (Please correct my if I misunderstood this). Should we add a new step to our CI and run mypy?

I think we should at least add this to ubuntu-latest py3.8. For more complete checks we could also go for ubuntu-latest py37-min-all-deps and upstream-dev.

h5py version 3.1 broke our tests. I pinned it to version 2.10 in #4569. We should therefore

• fix the issues
• unpin h5py again

The failures could be related to a change how strings are read: https://docs.h5py.org/en/latest/strings.html I am not sure if this has to be fixed in xarray or in h5necdf. I'd be happy if someone else took this one.

Failed tests:

Example failure:

```python traceback

assert_allclose(original, computed) E AssertionError: Left and right Dataset objects are not close E
E Differing coordinates: E L * dim3 (dim3) <U1 'a' 'b' 'c' 'd' 'e' 'f' 'g' 'h' 'i' 'j' E R * dim3 (dim3) object b'a' b'b' b'c' b'd' b'e' b'f' b'g' b'h' b'i' b'j'< ```

What happened:

Our doctests fail since numpy 1.20 came out:

https://github.com/pydata/xarray/pull/4760/checks?check_run_id=1818512841#step:8:69

What you expected to happen:

They don't ;-)

Minimal Complete Verifiable Example:

The following fails with numpy 1.20 while it converted np.NaN to an integer before (xarray.DataArray.pad at the bottom)

```python import numpy as np

x = np.arange(10) x = np.pad(x, 1, "constant", constant_values=np.nan) ``` requires numpy 1.20

Anything else we need to know?:

• that's probably related to https://numpy.org/doc/stable/release/1.20.0-notes.html#numpy-scalars-are-cast-when-assigned-to-arrays
• I asked if this behavior will stay: https://github.com/numpy/numpy/issues/16499#issuecomment-772342087
• One possibility is to add a check np.can_cast(constant_values.dtype, array.dtype) (or similar) for a better error message.

The behavior of xarray when assigning values with incompatible dtypes is a bit arbitrary. This is partly due to the behavior of numpy.... numpy 1.20 got a bit cleverer but still seems inconsistent at times... I am not sure what to do about this (and if we should actually be clever here).

1. Direct assignment (dupe of #4612)

```python import xarray as xr import numpy as np

arr = np.array([2])

arr[0] = np.nan

ValueError (since numpy 1.20)

arr[0:1] = np.array([np.nan])

-> array([-9223372036854775808])

da = xr.DataArray([5], dims="x")

da[0] = np.nan

<xarray.DataArray (x: 1)>

array([-9223372036854775808])

Dimensions without coordinates: x

(because this gets converted to da.variable._data[0:1, 0:1] = np.array([np.nan]) (approximately).

da[0] = 1.2345

casts constant_values to int

```

1. Via a numpy function (pad, shift, rolling)

pad

```python da.pad(x=1, constant_values=np.nan)

ValueError: cannot convert float NaN to integer

da.pad(x=1, constant_values=None)

casts da to float

da.pad(x=1, constant_values=1.5)

casts constant_values to int

```

To check:

• What does dask do in these cases?
• What does pandas do?
• What about str dtypes?

Would be nice if it did :-)

e.g. https://github.com/pydata/xarray/runs/1698988952

What happened:

I try to save the result of ds.resample("time": "Q-FEB").mean() with a 360_day calendar as netcdf. Thus, my first date is cftime.Datetime360Day(2000-02-30) (note the day). xarray then tries to use units='days since 2000-02-30' which fails with ValueError: day is out of range for month.

What you expected to happen:

The dataset can be saved.

Minimal Complete Verifiable Example:

```python import cftime import xarray as xr

time = xr.cftime_range("2000-02-30", "2001-01-01", freq="3M", calendar="360_day")

dates = np.asarray(time) reference_date = xr.coding.times.infer_datetime_units(dates)

'days since 2000-02-30 00:00:00.000000'

xr.coding.times.encode_cf_datetime(time)

ValueError

``` Traceback:

Anything else we need to know?:

This goes wrong here: https://github.com/pydata/xarray/blob/15537497136345ed67e9e8b089bcd4573df0b2ea/xarray/coding/times.py#L249-L250

A possible fix is to add the following lines:

python try: cftime._cftime._dateparse(reference_date) except ValueError: reference_date = type(dates[0])(dates[0].year, dates[0].month, 1) reference_date = xr.coding.times.format_cftime_datetime(reference_date)

To workaround set the encoding manually:

python encoding = {} encoding['time'] = {'units': 'days since 1850-01-01'} ds.to_netcdf(filename, encoding=encoding)

The windows tests seem to regularly time out after 60 min, which is annoying. All other tests seem to run about twice as fast... There is some stuff we can try (which may or may not help):

• [ ] switch to the newest windows image
• [ ] use mamba to resolve dependencies
• [ ] use matplotlib-base
• [ ] find the slowest tests using py.test xarray/tests/ --durations=100
• [ ] de-parametrize tests with a slow setup (if possible)
• [ ] reduce the number of xfails
• [ ] other ideas?

The Linux py36-bare-minimum will likely fail with a TypeError: 'NoneType' object is not callable due to python/importlib_metadata#270

This could be fixed by adding typing_extensions to the CI or maybe they fix that upstream.

We use mamba in the new CI:

https://github.com/pydata/xarray/blob/7152b41fa80a56db0ce88b241fbe4092473cfcf0/.github/workflows/upstream-dev-ci.yaml#L39

I just tested it and it is awesomely fast! However, it does not enforce strict channel priority, so packages from main get mixed in (see e.g. https://github.com/pydata/xarray/runs/1487637678#step:5:112 & search for main) & we don't want that. So I'd suggest to do:

bash mamba env update --override-channels -c conda-forge -f ci/requirements/py38.yml

(note there is a --strict-channel-priority option in mamba but it still installed packages from main when I tested it)

cc @andersy005: mybe you could add this to #4604?

See

(http://xarray.pydata.org/en/latest/generated/xarray.concat.html)

Can this be fixed? Or do we actually want to show both?

As discussed in the call today it would be nice to remove the upstream test from our ci and setup a "nightly" job to test against the development versions of pandas etc. This should lead to less failures unrelated to specific PRs. Question: how are we going to see that the upstream build failed? Having the failure in random PRs is not clean but very visible...

This could be done via github actions. See xpublish for an example setup: https://github.com/xarray-contrib/xpublish/tree/master/.github/workflows

xref #4313

Now that #4502 was fixed upstream we get a new issue in py38-upstream-dev (which I am pretty sure is unrelated as dask is not involved). xarray/tests/test_cftimeindex_resample.py::test_resample fails with:

python-traceback AttributeError: 'CFTimeIndex' object has no attribute '_id'

See: https://dev.azure.com/xarray/xarray/_build/results?buildId=4038&view=logs&jobId=603f3fdc-8af6-5e0a-f594-fa71bc949352&j=603f3fdc-8af6-5e0a-f594-fa71bc949352&t=51624cf6-228d-5319-1d6f-8cd30bcca2e7

The failure did not happen on the 14. Oct. (https://github.com/pydata/xarray/commit/db4f03e467d13229512f8f7924dc142db1b9486b the failures here are #4502) but it appeared on the 15. Oct. (https://github.com/pydata/xarray/commit/15537497136345ed67e9e8b089bcd4573df0b2ea)

Maybe a change in pandas? I have not looked at it closely - maybe @spencerkclark sees what's going on?

See ecmwf/cfgrib#167. This causes our doc build to fail (#4520). The tests do not fail because import cfgrib issues a ModuleNotFoundError therfore @requires_cfgrib skips the test...

todo - [ ] unpin docs again - [ ] it's probably quite a corner case but - should the tests fail? How would this be achieved...?

The DataArray tests are skipped in py38-all-but-dask - I have no idea why...

See

https://dev.azure.com/xarray/xarray/_build/results?buildId=4028&view=logs&j=15c5a299-0936-5372-ef76-f6800a28dcef&t=b8653be4-abae-5992-d733-6af037900ab6&l=108

test_dataarray.py should be just before test_dataset.py

Our upstream tests fail due to a change in dask. The likely culprit is dask/dask#6680 - dask.array.zeros_like does now take the meta keyword and thus it tries to coerce str to a bool which fails (and it didn't do before). The following should trigger the error:

```python import xarray as xr import dask

da = xr.DataArray(dask.array.array("")) xr.zeros_like(da, dtype=bool) `` Note thatzeros_likeis called inisnull` (which triggered the test failures):

https://github.com/pydata/xarray/blob/080caf4246fe2f4d6aa0c5dcb65a99b376fa669b/xarray/core/duck_array_ops.py#L100-L102

What happened:

```python /home/vsts/work/1/s/xarray/tests/test_duck_array_ops.py:499:

/home/vsts/work/1/s/xarray/testing.py:139: in compat_variable return a.dims == b.dims and (a._data is b._data or equiv(a.data, b.data)) /home/vsts/work/1/s/xarray/testing.py:36: in _data_allclose_or_equiv return duck_array_ops.array_equiv(arr1, arr2) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:246: in array_equiv flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2)) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:104: in isnull return zeros_like(data, dtype=bool) /home/vsts/work/1/s/xarray/core/duck_array_ops.py:56: in f return wrapped(args, *kwargs) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/creation.py:174: in zeros_like return zeros( /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/wrap.py:78: in wrap_func_shape_as_first_arg return Array(dsk, name, chunks, dtype=dtype, meta=kwargs.get("meta", None)) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/dask/array/core.py:1083: in new meta = meta_from_array(meta, dtype=dtype)

x = array('', dtype='<U1'), ndim = 0, dtype = dtype('bool')

def meta_from_array(x, ndim=None, dtype=None):
    """Normalize an array to appropriate meta object

    Parameters
    ----------
    x: array-like, callable
        Either an object that looks sufficiently like a Numpy array,
        or a callable that accepts shape and dtype keywords

    if dtype and meta.dtype != dtype:

      meta = meta.astype(dtype)

E ValueError: invalid literal for int() with base 10: ''

```

MCVE Code Sample

```python import xarray as xr import numpy as np

ds = xr.DataArray([1, 2], attrs=dict(tst="DataArray")).to_dataset(name="data") ds.attrs["tst"] = "Dataset" ds.map(np.mean, keep_attrs=True).data ```

returns python <xarray.DataArray 'data' ()> array(1.5)

Expected Output

python <xarray.DataArray 'data' ()> array(1.5) Attributes: tst: DataArray

Problem Description

Applying xr.Dataset.map(..., keep_attrs=True) does not retain the attributes of the DataArrays. Should it?

In constrast ds.mean(keep_attrs=True) retains DataArray-level attrs.

EDIT: corrected example

Output of xr.show_versions()

Matplotlib 3.3.1 broke the doc build so I pinned it to 3.3.0 - see #4340

The likely culprit is https://github.com/matplotlib/matplotlib/issues/18254 - thanks @jthielen for finding the matplotlib issue

What happened: da.sum(min_count=1) returns a TypeError if da has an integer dtype. Of course min_count is not necessary for integer data as it cannot contain NaN.

What you expected to happen: min_count should be ignored

Minimal Complete Verifiable Example:

python import xarray as xr da = xr.DataArray([[1, 2, 3], [4, 5, 6]]) da.sum(min_count=1)

Anything else we need to know?:

Full traceback

Environment:

Another follow up from #4060: quantile now emits a FutureWarning:

Minimal Complete Verifiable Example:

python xr.DataArray([1, 2, 3]).quantile(q=0.5)

~/.conda/envs/ipcc_ar6/lib/python3.7/site-packages/xarray/core/variable.py:1866: FutureWarning: ``output_sizes`` should be given in the ``dask_gufunc_kwargs`` parameter. It will be removed as direct parameter in a future version. kwargs={"q": q, "axis": axis, "interpolation": interpolation},

We should probably check the warnings in the test suite - there may be others.

I tried to use apply_ufunc for a function that takes input of unequal length and requires vectorize=True which resulted in a ValueError. I think the problem stems from the way np.vectorize is called.

MCVE Code Sample

```python import xarray as xr import scipy as sp import scipy.stats import numpy as np

create dataarrays of unequal length

ds = xr.tutorial.open_dataset("air_temperature")

da1 = ds.air da2 = ds.air.isel(time=slice(None, 50))

function that takes arguments of unequal length and requires vectorizing

def mannwhitneyu(x, y): _, p = sp.stats.mannwhitneyu(x, y) return p

test that the function takes arguments of unequal length

mannwhitneyu(da1.isel(lat=0, lon=0), da2.isel(lat=0, lon=0))

xr.apply_ufunc( mannwhitneyu, da1, da2, input_core_dims=[["time"], ["time"]], exclude_dims=set(["time"]), vectorize=True, )

```

Returns

python ValueError: inconsistent size for core dimension 'n': 50 vs 2920 Note: the error stems from numpy.

Expected Output

A DataArray.

Problem Description

I can reproduce the problem in pure numpy:

python vec_wrong = np.vectorize(mannwhitneyu, signature="(n),(n)->()", otypes=[np.float]) vec_wrong(da1.values.T, da2.values.T)

The correct result is returned when the signature is changed: python vec_correct = np.vectorize(mannwhitneyu, signature="(m),(n)->()", otypes=[np.float]) vec_correct(da1.values.T, da2.values.T) So I think the signature needs to be changed when exclude_dims are present.

Versions

Is your feature request related to a problem? Please describe.

sum with min_count errors when passing more than one dim:

python import xarray as xr da = xr.DataArray([[1., 2, 3], [4, 5, 6]]) da.sum(["dim_0", "dim_1"], min_count=1)

Describe the solution you'd like The logic to calculate the number of valid elements is here: https://github.com/pydata/xarray/blob/1be777fe725a85b8cc0f65a2bc41f4bc2ba18043/xarray/core/nanops.py#L35

I think this can be fixed by replacing

mask.shape[axis] with np.take(a.shape, axis).prod()

Additional context Potentially relevant for #4351

It would be nice to be able to use a level of a MultiIndex for plotting.

MCVE Code Sample

```python import numpy as np import xarray as xr

da = xr.DataArray( np.random.randn(10), dims="x", coords=dict( a=("x", np.arange(10, 20)), b=("x", np.arange(1, 11)) ) )

da = da.set_index(x=["a", "b"]) da This creates the following DataArraypython <xarray.DataArray (x: 10)> array([-1.34516338, 0.97644817, -0.24032189, -0.70112418, -0.8686898 , -0.55607078, 0.56618151, 1.62847463, 0.84947296, -0.5775504 ]) Coordinates: * x (x) MultiIndex - a (x) int64 10 11 12 13 14 15 16 17 18 19 - b (x) int64 1 2 3 4 5 6 7 8 9 10 ```

Is there a way to plot a line using one of the levels of the MultiIindex? python da.plot(x="a") returns python ValueError: x must be either None or one of ('x')

python da.plot() returns

python TypeError: Plotting requires coordinates to be numeric or dates of type np.datetime64, datetime.datetime, cftime.datetime or pd.Interval. (which makes sense). If da is a 2D Variable the error is python ValueError: x and y must be coordinate variables

Expected Output

A line plot

Versions

MCVE Code Sample

```python import numpy as np import xarray as xr

dta = xr.DataArray([1., 1., 1.]) wgt = xr.DataArray(np.array([1, 1, 0], dtype=np.bool))

dta.weighted(wgt).mean() ``` Returns

<xarray.DataArray ()> array(2.)

Expected Output

<xarray.DataArray ()> array(1.)

Problem Description

Passing a boolean array as weights to the weighted mean returns the wrong result because the weights are not properly normalized (in this case). Internally the sum_of_weights is calculated as

python xr.dot(dta.notnull(), wgt) i.e. the dot product of two boolean arrays. This yields: <xarray.DataArray ()> array(True)

We'll need to convert it to int or float: python xr.dot(dta.notnull(), wgt * 1) which is correct <xarray.DataArray ()> array(2)

Versions

MCVE Code Sample

Add arbitrary change to the file doc/pandas.rst

bash git add doc/pandas.rst git commit -m "test"

The pre-commit hook will fail.

Expected Output

the pre-commit hook to pass

Problem Description

running isort -rc doc/* will change the following files: bash modified: contributing.rst modified: howdoi.rst modified: internals.rst modified: io.rst modified: pandas.rst modified: quick-overview.rst unfortunately it does not behave properly and deletes/ changes arbitrary lines. Can the pre-commit hook be told to only run on *.py files? On the command line this would be isort -rc *.py

MCVE Code Sample

```python import numpy as np import xarray as xr

data = np.random.randn(10, 10) data = np.abs(data)

da = xr.DataArray(data)

returns a diverging colormap

da.plot(vmax=2, center=0, extend="both")

returns a sequential colormap

da.plot(levels=[-2, -1, 0, 1, 2], extend="both")

```

Expected Output

A diverging colormap, maybe?

Problem Description

I was surprised by getting the viridis colormap until I realised that my data must all be positive and the colormap is infered from the data and not from levels. However, when specifying the range via vmax=2, center=0 it is not inferred from the data.

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import xarray as xr import numpy as np x = np.arange(4.) ds = xr.DataArray(x, dims=('dim', )) ds.rolling(dim=3, center=True).mean()

RESULT: array([nan, 1., 2., nan])

ds.rolling(dim=3, center=True).construct('window').mean('window')

RESULT: array([0.5, 1. , 2. , 2.5])

```

Problem description

ds.rolling(...).mean() and ds.rolling(...).construct().mean() yields different results. Because mean does skipna=True per default.

Expected Output

I would expect both ways to yield the same result.

Output of xr.show_versions()

There is a bunch of operations that need to be conducted on a 1D array. It could be handy to have a convinience method that mimicks numpys flatten, i.e. works along the lines:

def flatten(dataarray, name='stacked'): return dataarray.stack(**{name: dataarray.dims})

What do you think?

It would be nice to be able to do ds.average() to compute weighted averages (e.g. for geo data). Of course this would require the axes to be in a predictable order. Or to give a weight per dimension...

Seaborn recently dropped the apionly entry point (in version 0.10.0).

There is a dedicated import_seaborn function https://github.com/pydata/xarray/blob/47a6fc0c9b796997996d79d32358585d08021b9c/xarray/plot/utils.py#L24

but it is not used in one place

https://github.com/pydata/xarray/blob/47a6fc0c9b796997996d79d32358585d08021b9c/xarray/plot/utils.py#L146

which now causes test failures

https://dev.azure.com/xarray/xarray/_build/results?buildId=2111&view=ms.vss-test-web.build-test-results-tab&runId=29632&paneView=debug&resultId=108176

Todo: double check if the supported versions of seaborn still require this workaround.

MCVE Code Sample

```python import xarray as xr import numpy as np

d1 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[0, 1, 2, 3])) d2 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[0, 1, 2, 3]))

note: different coords

d3 = xr.DataArray(np.arange(4), dims=["a"], coords=dict(a=[1, 2, 3, 4]))

(d1 * d2).sum() # -> array(14) xr.dot(d1, d2) # -> array(14)

(d2 * d3).sum() # -> array(8) xr.dot(d2, d3) # -> ValueError

```

Expected Output

python <xarray.DataArray ()> array(8)

Problem Description

The last statement results in an python ValueError: indexes along dimension 'a' are not equal because xr.apply_ufunc defaults to join='exact'. However, I think this should work - but maybe there is a good reason for this to fail?

This is a problem for #2922 (weighted operations) - I think it is fine for the weights and data to not align.

Fixing this may be as easy as specifying join='inner' in

https://github.com/pydata/xarray/blob/e0fd48052dbda34ee35d2491e4fe856495c9621b/xarray/core/computation.py#L1181-L1187

@fujiisoup

Output of xr.show_versions()

MCVE Code Sample

``` python import numpy as np import xarray as xr time = xr.cftime_range("2006-01-01", periods=2, calendar="360_day")

da = xr.DataArray(time, dims=["time"]) da.encoding["dtype"] = np.float da.to_netcdf("tst.nc", format="NETCDF4_CLASSIC")

ds = xr.open_dataset("tst.nc") ds.to_netcdf("tst2.nc", format="NETCDF4_CLASSIC") ```

yields: python ValueError: could not safely cast array from dtype int64 to int32

Or an example without to_netcdf:

```python import numpy as np import xarray as xr

time = xr.cftime_range("2006-01-01", periods=2, calendar="360_day")

da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan])

xr.backends.netcdf3.encode_nc3_variable(xr.conventions.encode_cf_variable(da)) ```

Expected Output

Xarray can save the dataset/ an xr.Variable.

Problem Description

If there is a time variable that can be encoded using integers only, but that has a _FillValue set to NaN, saving to_netcdf(name, format="NETCDF4_CLASSIC") fails. The problem is that xarray adds a (unnecessary) _FillValue when saving a file.

Note: if the time cannot be encoded using integers only, it works: ``` python da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan]) da.encoding["units"] = "days since 2006-01-01T12:00:00"

xr.backends.netcdf3.encode_nc3_variable(xr.conventions.encode_cf_variable(da))

```

Another note: when saving with NETCDF4 ``` python da = xr.DataArray(time, dims=["time"]) da.encoding["_FillValue"] = np.array([np.nan])

xr.backends.netCDF4_._encode_nc4_variable(xr.conventions.encode_cf_variable(da)) The following is returned: <xarray.Variable (time: 2)> array([0, 1]) Attributes: units: days since 2006-01-01 00:00:00.000000 calendar: proleptic_gregorian _FillValue: [-9223372036854775808] ```

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr da = xr.DataArray([0, 1, 2]) da.quantile(q=50)

ValueError: Percentiles must be in the range [0, 100] ```

Expected Output

python ValueError: Quantiles must be in the range [0, 1]

Problem Description

By wrapping np.nanpercentile (xref: #3559) we also get the numpy error. However, the error message is wrong as xarray needs it to be in 0..1.

BTW: thanks for #3559, makes my life easier!

Output of xr.show_versions()

Edit: uses nanpercentile internally.

MCVE Code Sample

python import xarray as xr i1 = xr.cftime_range("4991-01-01", periods=1) i2 = xr.cftime_range("7190-12-31", periods=1) i2 - i1

Expected Output

a timedelta

Problem Description

returns OverflowError: Python int too large to convert to C long. Originally I stumbled upon this when trying to open_mfdataset files from a long simulation (piControl). I did not figure out yet where this subtraction happens in open_mfdataset. (Opening the single files and using xr.concat works).

The offending lines are here

https://github.com/pydata/xarray/blob/40588dc38ddc2d573e3dc8c63b2e6533eb978656/xarray/coding/cftimeindex.py#L433

Ultimately this is probably a pandas problem as it tries to convert datetime.timedelta(days=803532) to '<m8[ns]'. pd.TimedeltaIndex has a (undocumented) dtype argument but I was not able to make anything else work (e.g. '<m8[D]').

@spencerkclark

Output of xr.show_versions()

MCVE Code Sample

```python import numpy as np import xarray as xr da_a = xr.DataArray(np.arange(3 * 2).reshape(3, 2), dims=['a', 'b']) da_b = xr.DataArray(np.arange(3 * 2 * 2).reshape(3, 2, 2), dims=['a', 'b', 'c'])

xr.dot(da_a, da_b, dims=None)

This sums over the common dimensions:python <xarray.DataArray (c: 2)> array([110, 125]) Dimensions without coordinates: c ```

To sum over all dimensions: ```python xr.dot(da_a, da_b, dims=['a', 'b', 'c'])

python <xarray.DataArray (c: 2)> array([110, 125]) Dimensions without coordinates: c ```

Problem Description

xr.dot with dims=None currently sums over all the common dimensions. However, there are cases when a sum over all dimensions is desired. E.g. xr.dot(da_a, da_b, dims=['a', 'b', 'c']) is a a memory efficient way to compute (da_a * da_b).sum() (if a and b don't share the same dimensions). This is currently used in #2922 (example).

Therefore I suggest to allow xr.dot(da_a, da_b, dims=xr.ALL_DIMS) as shortcut to sum over all dimensions.

I assume there is no intent to change the behavior of xr.dot(..., dims=None)? (As it is a bit in contrast to other functions that are mostly applied over all dimensions.)

@fujiisoup @max-sixty

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr import numpy as np

airtemps = xr.tutorial.load_dataset('air_temperature')

air = airtemps.air.isel(time=0)

levels = np.arange(225, 301, 25)

colors = ['#ffffb2', '#fecc5c', '#fd8d3c', '#e31a1c']

this does not work for me

air.plot.pcolormesh(levels=levels, colors=colors)

```

Expected Output

Should create a plot with the specified colors. According to the the docstring this should work. Or maybe I am doing something wrong here?

Problem Description

Instead I get the following error: ```python /usr/local/Miniconda3-envs/envs/2019/envs/iacpy3_2019/lib/python3.7/site-packages/xarray/plot/utils.py in _process_cmap_cbar_kwargs(func, kwargs, data) 683 # colors is only valid when levels is supplied or the plot is of type 684 # contour or contourf --> 685 if colors and (('contour' not in func.name) and (not levels)): 686 raise ValueError("Can only specify colors with contour or levels") 687

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ```

Instead I use the following, which works:

python air.plot.pcolormesh(levels=levels, cmap=colors)

Output of xr.show_versions()

Code Sample

```python import xarray as xr

a = xr.DataArray([1, 2], dims=('foo', ), name='a') b = xr.DataArray([3], dims=('bar', ), name='b')

objects = dict(a=a, b=b) xr.merge(objects)

```

Problem description

This returns an AttributeError, while the documentation states that a dict is a valid input.

``` python def merge(objects, compat='no_conflicts', join='outer'): """Merge any number of xarray objects into a single Dataset as variables.

Parameters
----------
objects : Iterable[Union[xarray.Dataset, xarray.DataArray, dict]]
    Merge together all variables from these objects. If any of them are
    DataArray objects, they must have a name.

...

```

Expected Output

Either the docs or the dict_like_objects loop should be changed.

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import xarray as xr import numpy as np

x = np.arange(100) * 365 x = xr.coding.times.decode_cf_datetime(x, 'days since 2400-01-01', calendar='proleptic_gregorian')

```

Problem description

xarray still throws an error when decoding out-of-bounds dates for proleptic_gregorian - should this be silenced?

Output of xr.show_versions()

Assume I have two datasets with the same lat/ lon grid. However, one has lon = 0...359and the other lon = -180...179. How can I wrap around one of them?

It would be nice to support nearest neighbour selection with a slice like so: ds.sel(lat=slice(30, 50), method='nearest'). Maybe the step has to be disallowed.

Or is there another possibility how to achieve this functionality?

use case: Often I look at a region with different data sets that don't have the same resolutions. It would be nice to be able to select the closest-matching region with an easy syntax.

Code Sample, a copy-pastable example if possible

```python

Your code here

import numpy as np import xarray as xr x = np.arange(10) da = xr.Dataset(data_vars=dict(data=('dim1', x)), coords=dict(dim1=('dim1', x), dim2=('dim2', x))) da.to_netcdf('tst.nc', format='NETCDF4_CLASSIC', unlimited_dims='dim1')

```

Problem description

This creates the error RuntimeError: NetCDF: NC_UNLIMITED size already in use. With format='NETCDF4' silently creates the dimensions d, i, m, and \1.

The correct syntax is unlimited_dims=['dim1'].

With format='NETCDF4_CLASSIC' and unlimited_dims=['dim1', 'dim2'], still raises the not-so-helpful NC_UNLIMITED error.

I only tested with netCDF4 as backend.

Expected Output

• better error message
• work with unlimited_dims='dim1'

Output of xr.show_versions()

Code Sample

```python

import numpy as np import xarray as xr

def test_units(units):

x = np.arange(10)
data = np.random.randn(10)

ds = xr.Dataset(data_vars=dict(data=('x', data)), coords=dict(x=('x', x)))
ds.data.attrs['units'] = units

ds.to_netcdf('tst.nc')

decoded = xr.open_dataset('tst.nc')

print(units.ljust(8), decoded.data.dtype)


ds.close()
decoded.close()

test_units('seconds') test_units('second')

test_units('minutes') test_units('minute')

test_units('days') test_units('day')

test_units('months') test_units('years') ```

Problem description

Returns: seconds timedelta64[ns] second float64 minutes timedelta64[ns] minute float64 days timedelta64[ns] day float64 months float64 years float64

Expected Output

I would expect type float for all of them. Or is this expected behaviour?

I have a dataset that reports 'consecutive dry days' and the dataset creator correctly set the units of the data to 'days', but I don't want this to be decoded (but the time axis should)....

Output of xr.show_versions()

python from xarray.plot.utils import import_seaborn _color_palette('Greys', 3) Returns 0.85, ... 0.58, ... 0.31, ...

if seaborn is installed, and

1.00, ... 0.58, ... 0.00, ...

otherwise

Problem description

the return value of _color_palette('Greys', 3) is different when seaborn is installed or not.

The relevant code is here: https://github.com/pydata/xarray/blob/6aa225f5dae9cc997e232c11a63072923c8c0238/xarray/plot/utils.py#L115 https://github.com/pydata/xarray/blob/6aa225f5dae9cc997e232c11a63072923c8c0238/xarray/plot/utils.py#L143

The same logic in seaborn https://github.com/mwaskom/seaborn/blob/0beede57152ce80ce1d4ef5d0c0f1cb61d118375/seaborn/palettes.py#L411

Intuitively I prefer the xarray solution because this uses the full range of colors which I find beneficial, however there may be a reason for this I'm not aware of.

Maybe @mwaskom will answer: mwaskom/seaborn#1372

Expected Output

Output of xr.show_versions()

~~

edit:

sorry I pressed the button to early

It would be nice if where accepts an other argument:

def where(self, cond, other=np.NaN): pass

How would I calculate percentiles over a certain dimension?

ds.groupby('time.month').apply(np.percentile, q=5)

Does not work.

Not sure if this is possible to specify in the setup.py as bottleneck is not required. However, if bottleneck is used it should probably be in version 1.0. For version 0.8.0 I get the following error:

>>> move_mean() got an unexpected keyword argument 'min_count'

Feel free to close.

If I have a netCDF file that has invalid timesteps from the 'future', it is wrongly converted to datetime64[ns].

``` import netCDF4 as nc import numpy as np import xarray as xr

create netCDF file

ncf = nc.Dataset('test_future.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1850-01-01 00:00:00' ncf.variables['time'].calendar = 'standard' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_future.nc')

this works

ds

ds.time is a datetime64[ns] object

this fails

ds.time ```

If I choose chalendar='noleap' the dates wrap around!

``` ncf = nc.Dataset('test_future_noleap.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1850-01-01 00:00:00' ncf.variables['time'].calendar = 'noleap' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_future_noleap.nc')

after 2262 they go back to 1678!

ds.time ```

If my 'invalid' time is from the 'past' it works as expected:

``` ncf = nc.Dataset('test_past.nc', 'w') ncf.createDimension('time') ncf.createVariable('time', np.int, dimensions=('time')) ncf.variables['time'].units = 'days since 1000-01-01 00:00:00' ncf.variables['time'].calendar = 'standard' ncf.variables['time'][:] = np.arange(850) * 365 ncf.close()

open with xr

ds = xr.open_dataset('test_past.nc')

this works

ds

ds.time is a object

ds.time ```

Sometimes it happens that you have two grids that are equal up to precision or "for practical purposes". xarray does not align them. Is there a possibility to still get them aligned (and not having to copy the coordinates from one to the other)?

Examples: - CESM output from the land model and the atmosphere model. - I have a file with a region definition on a fine grid. If I regrid that to the model resolution with cdo the grids are equal up to ~1*10**-4.

reindex can cause problems as long as the data is not loaded

``` import xarray as xr data = xr.DataArray([1, 2, 3], dims='x').to_dataset('name') data.to_netcdf('tst.nc')

works fine

a = xr.open_dataset('tst.nc') a.load() a.reindex(x=[2, 0, 1])

problem

b = xr.open_dataset('tst.nc') b.reindex(x=[2, 0, 1]) ```

When specifying a negative vmin and a positive vmax the viridis colormap is used. Is this intentional?

``` import xarray as xr import matplotlib.pyplot as plt

airtemps = xr.tutorial.load_dataset('air_temperature') air2d = airtemps.air.isel(time=500) - 273.15

diverging colormap

air2d.plot.pcolormesh() plt.show()

viridis

air2d.plot.pcolormesh(vmin=-24, vmax=24) plt.show()

------------------------------------------------------------

the solution:

diverging colormap

air2d.plot.pcolormesh(vmin=-24, center=0) plt.show() ```

EDIT: solution and working example

Now that we are at it... Why don't DataArrays have virtual variables?

It is quite convenient to select a whole year ds.sel(time='1991'). However, sel should not be used to assign data. Thus you can not do ds.sel(time='1991').coords['time'] = time and ds[dict(time='1991')] does not work.

1) Should ds[dict(time='1991')] work?

2) A possible workaround is:

idx = xray.core.indexing.remap_label_indexers(ds, {'time' : '1999'}) ds[idx] = time

Should this be added to the documentation or is this case too uncommon?

Nearest neighbor lookups does not seem to work:

import xray xray.__version__

0.4.1

data = xray.DataArray([1, 2, 3], dims='x') data.sel(x=[1.1, 1.9], method='nearest')

ValueError: not all values found in index 'x'

Example from: http://xray.readthedocs.org/en/latest/indexing.html#nearest-neighbor-lookups

(pandas version 0.16.1)

Once I take a mean over virtual variables, they are not available any more.

import pandas as pd import numpy as np import xray t = pd.date_range('2000-01-01', '2000-12-31', freq='6H') x = np.random.rand(*t.shape) time = xray.DataArray(t, name='t', dims='time') ts = xray.Dataset({'x' : ('time', x), 'time' : time}) ts_mean = ts.groupby('time.date').mean() ts_mean.virtual_variables

Is this intended behaviour? And could I get them back somehow?

Virtual variables such as time.date can not be accessed by dot notation.

``` In [81]: ts.time.date

AttributeError Traceback (most recent call last) <ipython-input-81-4206b510e310> in <module>() ----> 1 ts.time.date

/home/mathause/.local/lib/python2.7/site-packages/xray/core/common.pyc in getattr(self, name) 115 pass 116 raise AttributeError("%r object has no attribute %r" % --> 117 (type(self).name, name)) 118 119 def dir(self):

AttributeError: 'DataArray' object has no attribute 'date' ```

Compare

ts.__getattr__('time.date')

So maybe time.date is not a useful name. time_date? Or don't show these attributes to the user?