id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1574694462,I_kwDOAMm_X85d2-4-,7513,"intermittent failures with h5netcdf, h5py on macos",2448579,closed,0,,,5,2023-02-07T16:58:43Z,2024-04-28T23:35:21Z,2024-04-28T23:35:21Z,MEMBER,,,,"### What is your issue? cc @hmaarrfk @kmuehlbauer Passed: https://github.com/pydata/xarray/actions/runs/4115923717/jobs/7105298426 Failed: https://github.com/pydata/xarray/actions/runs/4115946392/jobs/7105345290 Versions: ``` h5netcdf 1.1.0 pyhd8ed1ab_0 conda-forge h5py 3.8.0 nompi_py310h5555e59_100 conda-forge hdf4 4.2.15 h7aa5921_5 conda-forge hdf5 1.12.2 nompi_h48135f9_101 conda-forge ``` ``` =================================== FAILURES =================================== _____________ test_open_mfdataset_manyfiles[h5netcdf-20-True-5-5] ______________ [gw1] darwin -- Python 3.10.9 /Users/runner/micromamba-root/envs/xarray-tests/bin/python readengine = 'h5netcdf', nfiles = 20, parallel = True, chunks = 5 file_cache_maxsize = 5 @requires_dask @pytest.mark.filterwarnings(""ignore:use make_scale(name) instead"") def test_open_mfdataset_manyfiles( readengine, nfiles, parallel, chunks, file_cache_maxsize ): # skip certain combinations skip_if_not_engine(readengine) if ON_WINDOWS: pytest.skip(""Skipping on Windows"") randdata = np.random.randn(nfiles) original = Dataset({""foo"": (""x"", randdata)}) # test standard open_mfdataset approach with too many files with create_tmp_files(nfiles) as tmpfiles: writeengine = readengine if readengine != ""pynio"" else ""netcdf4"" # split into multiple sets of temp files for ii in original.x.values: subds = original.isel(x=slice(ii, ii + 1)) if writeengine != ""zarr"": subds.to_netcdf(tmpfiles[ii], engine=writeengine) else: # if writeengine == ""zarr"": subds.to_zarr(store=tmpfiles[ii]) # check that calculation on opened datasets works properly > with open_mfdataset( tmpfiles, combine=""nested"", concat_dim=""x"", engine=readengine, parallel=parallel, chunks=chunks if (not chunks and readengine != ""zarr"") else ""auto"", ) as actual: /Users/runner/work/xarray/xarray/xarray/tests/test_backends.py:3267: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /Users/runner/work/xarray/xarray/xarray/backends/api.py:991: in open_mfdataset datasets, closers = dask.compute(datasets, closers) /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/base.py:599: in compute results = schedule(dsk, keys, **kwargs) /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/threaded.py:89: in get results = get_async( /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/local.py:511: in get_async raise_exception(exc, tb) /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/local.py:319: in reraise raise exc /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/local.py:224: in execute_task result = _execute_task(task, data) /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/core.py:119: in _execute_task return func(*(_execute_task(a, cache) for a in args)) /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/dask/utils.py:72: in apply return func(*args, **kwargs) /Users/runner/work/xarray/xarray/xarray/backends/api.py:526: in open_dataset backend_ds = backend.open_dataset( /Users/runner/work/xarray/xarray/xarray/backends/h5netcdf_.py:417: in open_dataset ds = store_entrypoint.open_dataset( /Users/runner/work/xarray/xarray/xarray/backends/store.py:32: in open_dataset vars, attrs = store.load() /Users/runner/work/xarray/xarray/xarray/backends/common.py:129: in load (_decode_variable_name(k), v) for k, v in self.get_variables().items() /Users/runner/work/xarray/xarray/xarray/backends/h5netcdf_.py:220: in get_variables return FrozenDict( /Users/runner/work/xarray/xarray/xarray/core/utils.py:471: in FrozenDict return Frozen(dict(*args, **kwargs)) /Users/runner/work/xarray/xarray/xarray/backends/h5netcdf_.py:221: in (k, self.open_store_variable(k, v)) for k, v in self.ds.variables.items() /Users/runner/work/xarray/xarray/xarray/backends/h5netcdf_.py:200: in open_store_variable elif var.compression is not None: /Users/runner/micromamba-root/envs/xarray-tests/lib/python3.10/site-packages/h5netcdf/core.py:394: in compression return self._h5ds.compression _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <[AttributeError(""'NoneType' object has no attribute '_root'"") raised in repr()] Variable object at 0x151378970> @property def _h5ds(self): # Always refer to the root file and store not h5py object # subclasses: > return self._root._h5file[self._h5path] E AttributeError: 'NoneType' object has no attribute '_h5file' ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7513/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2136709010,I_kwDOAMm_X85_W5eS,8753,Lazy Loading with `DataArray` vs. `Variable`,2448579,closed,0,,,0,2024-02-15T14:42:24Z,2024-04-04T16:46:54Z,2024-04-04T16:46:54Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/8751
Originally posted by **ilan-gold** February 15, 2024 My goal is to get a dataset from [custom io-zarr backend lazy-loaded](https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#how-to-support-lazy-loading). But when I declare a `DataArray` based on the `Variable` which uses `LazilyIndexedArray`, everything is read in. Is this expected? I specifically don't want to have to use dask if possible. I have seen https://github.com/aurghs/xarray-backend-tutorial/blob/main/2.Backend_with_Lazy_Loading.ipynb but it's a little bit different. While I have a custom backend array inheriting from `ZarrArrayWrapper`, this example using `ZarrArrayWrapper` directly still highlights the same unexpected behavior of everything being read in. ```python import zarr import xarray as xr from tempfile import mkdtemp import numpy as np from pathlib import Path from collections import defaultdict class AccessTrackingStore(zarr.DirectoryStore): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._access_count = {} self._accessed = defaultdict(set) def __getitem__(self, key): for tracked in self._access_count: if tracked in key: self._access_count[tracked] += 1 self._accessed[tracked].add(key) return super().__getitem__(key) def get_access_count(self, key): return self._access_count[key] def set_key_trackers(self, keys_to_track): if isinstance(keys_to_track, str): keys_to_track = [keys_to_track] for k in keys_to_track: self._access_count[k] = 0 def get_subkeys_accessed(self, key): return self._accessed[key] orig_path = Path(mkdtemp()) z = zarr.group(orig_path / ""foo.zarr"") z['array'] = np.random.randn(1000, 1000) store = AccessTrackingStore(orig_path / ""foo.zarr"") store.set_key_trackers(['array']) z = zarr.group(store) arr = xr.backends.zarr.ZarrArrayWrapper(z['array']) lazy_arr = xr.core.indexing.LazilyIndexedArray(arr) # just `.zarray` var = xr.Variable(('x', 'y'), lazy_arr) print('Variable read in ', store.get_subkeys_accessed('array')) # now everything is read in da = xr.DataArray(var) print('DataArray read in ', store.get_subkeys_accessed('array')) ```
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8753/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2066510805,I_kwDOAMm_X857LHPV,8589,"Don't overwrite indexes for region writes, always",2448579,closed,0,,,2,2024-01-04T23:52:18Z,2024-03-27T16:24:37Z,2024-03-27T16:24:36Z,MEMBER,,,,"### What happened? Currently we don't overwrite indexes when `region=""auto""` https://github.com/pydata/xarray/blob/e6ccedb56ed4bc8d0b7c1f16ab325795330fb19a/xarray/backends/api.py#L1769-L1770 I propose we do this for all region writes and completely disallow modifying indexes with a region write. This would match the `map_blocks` model, where all indexes are specified in the `template` and no changes by the mapped function are allowed. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8589/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2188936276,I_kwDOAMm_X86CeIRU,8843,Get ready for pandas 3 copy-on-write,2448579,closed,0,,,2,2024-03-15T15:51:36Z,2024-03-18T16:00:14Z,2024-03-18T16:00:14Z,MEMBER,,,,"### What is your issue? This line fails with `pd.set_options(""mode.copy_on_write"", True)` https://github.com/pydata/xarray/blob/c9d3084e98d38a7a9488380789a8d0acfde3256f/xarray/tests/__init__.py#L329 We'll need to fix this before Pandas 3 is released in April: https://github.com/pydata/xarray/blob/c9d3084e98d38a7a9488380789a8d0acfde3256f/xarray/tests/__init__.py#L329 Here's a test ```python def example(): obj = Dataset() obj[""dim2""] = (""dim2"", 0.5 * np.arange(9)) obj[""time""] = (""time"", pd.date_range(""2000-01-01"", periods=20) print({k: v.data.flags for k, v in obj.variables.items()}) return obj example() pd.set_options(""mode.copy_on_write"", True) example() ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8843/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2098659703,I_kwDOAMm_X859FwF3,8659,renaming index variables with `rename_vars` seems buggy,2448579,closed,0,,,1,2024-01-24T16:35:18Z,2024-03-15T19:21:51Z,2024-03-15T19:21:51Z,MEMBER,,,,"### What happened? (xref #8658) I'm not sure what the expected behaviour is here: ```python import xarray as xr import numpy as np from xarray.testing import _assert_internal_invariants ds = xr.Dataset() ds.coords[""1""] = (""1"", np.array([1], dtype=np.uint32)) ds[""1_""] = (""1"", np.array([1], dtype=np.uint32)) ds = ds.rename_vars({""1"": ""0""}) ds ``` It looks like this sequence of operations creates a default index But then ```python from xarray.testing import _assert_internal_invariants _assert_internal_invariants(ds, check_default_indexes=True) ``` fails with ``` ... File ~/repos/xarray/xarray/testing/assertions.py:301, in _assert_indexes_invariants_checks(indexes, possible_coord_variables, dims, check_default) 299 if check_default: 300 defaults = default_indexes(possible_coord_variables, dims) --> 301 assert indexes.keys() == defaults.keys(), (set(indexes), set(defaults)) 302 assert all(v.equals(defaults[k]) for k, v in indexes.items()), ( 303 indexes, 304 defaults, 305 ) AssertionError: ({'0'}, set()) ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8659/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2184871888,I_kwDOAMm_X86COn_Q,8830,"failing tests, all envs",2448579,closed,0,,,1,2024-03-13T20:56:34Z,2024-03-15T04:06:04Z,2024-03-15T04:06:04Z,MEMBER,,,,"### What happened? All tests are failing because of an error in `create_test_data` ``` from xarray.tests import create_test_data create_test_data() ``` ``` --------------------------------------------------------------------------- AssertionError Traceback (most recent call last) Cell In[3], line 2 1 from xarray.tests import create_test_data ----> 2 create_test_data() File [~/repos/xarray/xarray/tests/__init__.py:329](http://localhost:8888/lab/workspaces/auto-P/tree/repos/devel/arraylake/~/repos/xarray/xarray/tests/__init__.py#line=328), in create_test_data(seed, add_attrs, dim_sizes) 327 obj.coords[""numbers""] = (""dim3"", numbers_values) 328 obj.encoding = {""foo"": ""bar""} --> 329 assert all(var.values.flags.writeable for var in obj.variables.values()) 330 return obj AssertionError: ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8830/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1308371056,I_kwDOAMm_X85N_Chw,6806,"New alignment option: ""exact"" without broadcasting OR Turn off automatic broadcasting",2448579,closed,0,,,9,2022-07-18T18:43:31Z,2024-03-13T15:36:35Z,2024-03-13T15:36:35Z,MEMBER,,,,"### Is your feature request related to a problem? If we have two objects with dims `x` and `x1`, then `xr.align(..., join=""exact"")` will pass because these dimensions are broadcastable. I'd like a stricter option (`join=""strict""`?) that disallows broadcasting. ### Describe the solution you'd like ```python xr.align( xr.DataArray([1], dims=""x""), xr.DataArray([1], dims=""x1""), join=""strict"", ) ``` would raise an error. It'd be nice to have this as a built-in option so we can use ``` python with xr.set_options(arithmetic_join=""strict""): ... ``` ### Describe alternatives you've considered An alternative would be to allow control over automatic broadcasting through the `set_options` context manager., but that seems like it would be more complicated to implement. ### Additional context This turns up in staggered grid calculations with xgcm where it is easy to mistakenly construct very high-dimensional arrays because of automatic broadcasting.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6806/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2135011804,I_kwDOAMm_X85_QbHc,8748,release v2024.02.0,2448579,closed,0,14808389,,0,2024-02-14T19:08:38Z,2024-02-18T22:52:15Z,2024-02-18T22:52:15Z,MEMBER,,,,"### What is your issue? Thanks to @keewis for volunteering at today's meeting :()","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8748/reactions"", ""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2064313690,I_kwDOAMm_X857Cu1a,8580,add py3.12 CI and update pyproject.toml,2448579,closed,0,,,2,2024-01-03T16:26:47Z,2024-01-17T21:54:13Z,2024-01-17T21:54:13Z,MEMBER,,,,"### What is your issue? We haven't done this yet! https://github.com/pydata/xarray/blob/d87ba61c957fc3af77251ca6db0f6bccca1acb82/pyproject.toml#L11-L15","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8580/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2086607437,I_kwDOAMm_X858XxpN,8616, new release 2024.01.0,2448579,closed,0,,,0,2024-01-17T17:03:20Z,2024-01-17T19:21:12Z,2024-01-17T19:21:12Z,MEMBER,,,,"### What is your issue? Thanks @TomNicholas for volunteering to drive this release!","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8616/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2064420057,I_kwDOAMm_X857DIzZ,8581,bump min versions,2448579,closed,0,,,0,2024-01-03T17:45:10Z,2024-01-05T16:13:16Z,2024-01-05T16:13:15Z,MEMBER,,,,"### What is your issue? Looks like we can bump a number of min versions: ``` Package Required Policy Status ----------------- -------------------- -------------------- ------ cartopy 0.20 (2021-09-17) 0.21 (2022-09-10) < dask-core 2022.7 (2022-07-08) 2022.12 (2022-12-02) < distributed 2022.7 (2022-07-08) 2022.12 (2022-12-02) < flox 0.5 (2022-05-03) 0.6 (2022-10-12) < iris 3.2 (2022-02-15) 3.4 (2022-12-01) < matplotlib-base 3.5 (2021-11-18) 3.6 (2022-09-16) < numba 0.55 (2022-01-14) 0.56 (2022-09-28) < numpy 1.22 (2022-01-03) 1.23 (2022-06-23) < packaging 21.3 (2021-11-18) 22.0 (2022-12-08) < pandas 1.4 (2022-01-22) 1.5 (2022-09-19) < scipy 1.8 (2022-02-06) 1.9 (2022-07-30) < seaborn 0.11 (2020-09-08) 0.12 (2022-09-06) < typing_extensions 4.3 (2022-07-01) 4.4 (2022-10-07) < zarr 2.12 (2022-06-23) 2.13 (2022-09-27) < ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8581/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1989588884,I_kwDOAMm_X852lreU,8448,mypy 1.7.0 raising errors,2448579,closed,0,,,0,2023-11-12T21:41:43Z,2023-12-01T22:02:22Z,2023-12-01T22:02:22Z,MEMBER,,,,"### What happened? ``` xarray/namedarray/core.py:758: error: Value of type Never is not indexable [index] xarray/core/alignment.py:684: error: Unused ""type: ignore"" comment [unused-ignore] xarray/core/alignment.py:1156: error: Unused ""type: ignore"" comment [unused-ignore] xarray/core/dataset.py: note: In member ""sortby"" of class ""Dataset"": xarray/core/dataset.py:7967: error: Incompatible types in assignment (expression has type ""tuple[Alignable, ...]"", variable has type ""tuple[DataArray, ...]"") [assignment] xarray/core/dataset.py:7979: error: ""Alignable"" has no attribute ""isel"" [attr-defined] ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8448/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1615596004,I_kwDOAMm_X85gTAnk,7596,illustrate time offset arithmetic,2448579,closed,0,,,2,2023-03-08T16:54:15Z,2023-11-29T01:31:45Z,2023-11-29T01:31:45Z,MEMBER,,,,"### Is your feature request related to a problem? We should document changing the time vector using pandas date offsets [here](https://docs.xarray.dev/en/stable/user-guide/time-series.html#time-series-data) This is particularly useful for centering the time stamps after a resampling operation. Related: - CFTime offsets: https://github.com/pydata/xarray/issues/5687 - `loffset` deprecation: https://github.com/pydata/xarray/pull/7444 ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7596/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1672288892,I_kwDOAMm_X85jrRp8,7764,Support opt_einsum in xr.dot,2448579,closed,0,,,7,2023-04-18T03:29:48Z,2023-10-28T03:31:06Z,2023-10-28T03:31:06Z,MEMBER,,,,"### Is your feature request related to a problem? Shall we support [opt_einsum](https://dgasmith.github.io/opt_einsum/) as an optional backend for `xr.dot`? `opt_einsum.contract` is a drop-in replacement for `np.einsum` so this monkey-patch works today ``` xr.core.duck_array_ops.einsum = opt_einsum.contract ``` ### Describe the solution you'd like Add a `backend` kwarg with options `""numpy""` and `""opt_einsum""`, with the default being `""numpy""` ### Describe alternatives you've considered We could create a new package but it seems a bit silly. ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7764/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1908084109,I_kwDOAMm_X85xuw2N,8223,release 2023.09.0,2448579,closed,0,,,6,2023-09-22T02:29:30Z,2023-09-26T08:12:46Z,2023-09-26T08:12:46Z,MEMBER,,,,"We've accumulated a nice number of changes. Can someone volunteer to do a release in the next few days? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8223/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1175093771,I_kwDOAMm_X85GCoIL,6391,apply_ufunc and Datasets with variables without the core dimension,2448579,closed,0,,,5,2022-03-21T09:13:02Z,2023-09-17T08:20:15Z,2023-09-17T08:20:14Z,MEMBER,,,,"### Is your feature request related to a problem? Consider this example ```python ds = xr.Dataset({""a"": (""x"", [1, 2, 3]), ""b"": (""y"", [1, 2, 3])}) xr.apply_ufunc(np.mean, ds, input_core_dims=[[""x""]]) ``` This raises ``` ValueError: operand to apply_ufunc has required core dimensions ['x'], but some of these dimensions are absent on an input variable: ['x'] ``` because core dimension `x` is missing on variable `b`. This behaviour makes it annoying to use `apply_ufunc` on Datasets. ### Describe the solution you'd like Add a new kwarg to `apply_ufunc` called `missing_core_dim` that controls how to handle variables without *all* input core dimensions. This kwarg could take one of two values: 1. `""raise""` - raise an error, current behaviour 2. `""copy""` - skip applying the function and copy the variable from input to output. 3. `""drop""`- skip applying the function and drop the variable. ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6391/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1874695065,I_kwDOAMm_X85vvZOZ,8125,failing tests with pandas 2.1,2448579,closed,0,,,10,2023-08-31T02:42:32Z,2023-09-15T13:12:02Z,2023-09-15T13:12:02Z,MEMBER,,,,"### What happened? See https://github.com/pydata/xarray/pull/8101 ``` FAILED xarray/tests/test_missing.py::test_interpolate_pd_compat - ValueError: 'fill_value' is not a valid keyword for DataFrame.interpolate FAILED xarray/tests/test_missing.py::test_interpolate_pd_compat_non_uniform_index - ValueError: 'fill_value' is not a valid keyword for DataFrame.interpolate ``` and this doctest ``` FAILED xarray/core/dataarray.py::xarray.core.dataarray.DataArray.to_unstacked_dataset ``` @pydata/xarray can someone take a look please?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8125/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1812504689,I_kwDOAMm_X85sCKBx,8006,Fix documentation about datetime_unit of xarray.DataArray.differentiate,2448579,closed,0,,,0,2023-07-19T18:31:10Z,2023-09-01T09:37:15Z,2023-09-01T09:37:15Z,MEMBER,,,,"Should say that `Y` and `M` cannot be supported with `datetime64` ### Discussed in https://github.com/pydata/xarray/discussions/8000
Originally posted by **jesieleo** July 19, 2023 I have a piece of data that looks like this ``` Dimensions: (time: 612, LEV: 15, latitude: 20, longitude: 357) Coordinates: * time (time) datetime64[ns] 1960-01-15 1960-02-15 ... 2010-12-15 * LEV (LEV) float64 5.01 15.07 25.28 35.76 ... 149.0 171.4 197.8 229.5 * latitude (latitude) float64 -4.75 -4.25 -3.75 -3.25 ... 3.75 4.25 4.75 * longitude (longitude) float64 114.2 114.8 115.2 115.8 ... 291.2 291.8 292.2 Data variables: u (time, LEV, latitude, longitude) float32 ... Attributes: (12/30) cdm_data_type: Grid Conventions: COARDS, CF-1.6, ACDD-1.3 creator_email: chepurin@umd.edu creator_name: APDRC creator_type: institution creator_url: https://www.atmos.umd.edu/~ocean/ ... ... standard_name_vocabulary: CF Standard Name Table v29 summary: Simple Ocean Data Assimilation (SODA) soda po... time_coverage_end: 2010-12-15T00:00:00Z time_coverage_start: 1983-01-15T00:00:00Z title: SODA soda pop2.2.4 [TIME][LEV][LAT][LON] Westernmost_Easting: 118.25 ``` when i try to use xarray.DataArray.differentiate `data.u.differentiate('time',datetime_unit='M')` will appear ``` Traceback (most recent call last): File """", line 1, in File ""D:\Anaconda3\lib\site-packages\xarray\core\dataarray.py"", line 3609, in differentiate ds = self._to_temp_dataset().differentiate(coord, edge_order, datetime_unit) File ""D:\Anaconda3\lib\site-packages\xarray\core\dataset.py"", line 6372, in differentiate coord_var = coord_var._to_numeric(datetime_unit=datetime_unit) File ""D:\Anaconda3\lib\site-packages\xarray\core\variable.py"", line 2428, in _to_numeric numeric_array = duck_array_ops.datetime_to_numeric( File ""D:\Anaconda3\lib\site-packages\xarray\core\duck_array_ops.py"", line 466, in datetime_to_numeric array = array / np.timedelta64(1, datetime_unit) TypeError: Cannot get a common metadata divisor for Numpy datatime metadata [ns] and [M] because they have incompatible nonlinear base time units. ``` Would you please told me is this a BUG?
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1603957501,I_kwDOAMm_X85fmnL9,7573,Add optional min versions to conda-forge recipe (`run_constrained`),2448579,closed,0,,,4,2023-02-28T23:12:15Z,2023-08-21T16:12:34Z,2023-08-21T16:12:21Z,MEMBER,,,,"### Is your feature request related to a problem? I opened this PR to add minimum versions for our optional dependencies: https://github.com/conda-forge/xarray-feedstock/pull/84/files to prevent issues like #7467 I think we'd need a policy to choose which ones to list. Here's the current list: ``` run_constrained: - bottleneck >=1.3 - cartopy >=0.20 - cftime >=1.5 - dask-core >=2022.1 - distributed >=2022.1 - flox >=0.5 - h5netcdf >=0.13 - h5py >=3.6 - hdf5 >=1.12 - iris >=3.1 - matplotlib-base >=3.5 - nc-time-axis >=1.4 - netcdf4 >=1.5.7 - numba >=0.55 - pint >=0.18 - scipy >=1.7 - seaborn >=0.11 - sparse >=0.13 - toolz >=0.11 - zarr >=2.10 ``` Some examples to think about: 1. `iris` seems like a bad one to force. It seems like people might use Iris and Xarray independently and Xarray shouldn't force a minimum version. 2. For backends, I arbitrarily kept `netcdf4`, `h5netcdf` and `zarr`. 3. It seems like we should keep array types: so `dask`, `sparse`, `pint`. ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7573/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1642299599,I_kwDOAMm_X85h44DP,7683,automatically chunk in groupby binary ops,2448579,closed,0,,,0,2023-03-27T15:14:09Z,2023-07-27T16:41:35Z,2023-07-27T16:41:34Z,MEMBER,,,,"### What happened? From https://discourse.pangeo.io/t/xarray-unable-to-allocate-memory-how-to-size-up-problem/3233/4 Consider ``` python # ds is dataset with big dask arrays mean = ds.groupby(""time.day"").mean() mean.to_netcdf() mean = xr.open_dataset(...) ds.groupby(""time.day"") - mean ``` In `GroupBy._binary_op` https://github.com/pydata/xarray/blob/39caafae4452f5327a7cd671b18d4bb3eb3785ba/xarray/core/groupby.py#L616 we will eagerly construct `other` that is of the same size as `ds`. ### What did you expect to happen? I think the only solution is to automatically chunk if `ds` has dask arrays, and `other` (or `mean`) isn't backed by dask arrays. A chunk size of `1` seems sensible. ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7683/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1789989152,I_kwDOAMm_X85qsREg,7962,Better chunk manager error,2448579,closed,0,,,4,2023-07-05T17:27:25Z,2023-07-24T22:26:14Z,2023-07-24T22:26:13Z,MEMBER,,,,"### What happened? I just ran in to this error in an environment without dask. ``` TypeError: Could not find a Chunk Manager which recognises type ``` I think we could easily recommend the user to install a package that provides `dask` by looking at `type(array).__name__`. This would make the message a lot friendlier ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7962/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1797636782,I_kwDOAMm_X85rJcKu,7976,Explore updating colormap code,2448579,closed,0,,,0,2023-07-10T21:51:30Z,2023-07-11T13:49:54Z,2023-07-11T13:49:53Z,MEMBER,,,,"### What is your issue? See https://github.com/matplotlib/matplotlib/issues/16296 Looks like the MPL API may have advanced enough that we can delete some of our use of private attributes.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7976/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1692597701,I_kwDOAMm_X85k4v3F,7808,Default behaviour of `min_count` wrong with flox,2448579,closed,0,,,0,2023-05-02T15:04:11Z,2023-05-10T02:39:45Z,2023-05-10T02:39:45Z,MEMBER,,,,"### What happened? ```python with xr.set_options(display_style=""text"", use_flox=False): with xr.set_options(use_flox=False): display( xr.DataArray( data=np.array([np.nan, 1, 1, np.nan, 1, 1]), dims=""x"", coords={""labels"": (""x"", np.array([1, 2, 3, 1, 2, 3]))}, ) .groupby(""labels"") .sum() ) with xr.set_options(use_flox=True): display( xr.DataArray( data=np.array([np.nan, 1, 1, np.nan, 1, 1]), dims=""x"", coords={""labels"": (""x"", np.array([1, 2, 3, 1, 2, 3]))}, ) .groupby(""labels"") .sum() ) ``` ``` # without flox array([0., 2., 2.]) Coordinates: * labels (labels) int64 1 2 3 # with flox array([nan, 2., 2.]) Coordinates: * labels (labels) int64 1 2 3 ``` ### What did you expect to happen? The same answer. We should set `min_count=0` when `min_count is None` ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7808/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1654022522,I_kwDOAMm_X85ilmF6,7716,bad conda solve with pandas 2,2448579,closed,0,,,18,2023-04-04T14:37:58Z,2023-04-16T17:57:27Z,2023-04-13T17:56:34Z,MEMBER,,,,"### What happened? Pandas 2 is out. We have a `pandas<2` pin for our latest release, but `mamba` is now returning `xarray=2023.1.0` and `pandas=2.0` which is making [cf-xarray](https://github.com/xarray-contrib/cf-xarray/actions/runs/4603068761/jobs/8132721316?pr=431) and [flox](https://github.com/xarray-contrib/flox/actions/runs/4603066011/jobs/8132714566?pr=229) tests fail. It looks like any project that tests `resample` without pinning pandas will fail. I opened the issue here for visibility. It seems we might need a repodata patch to disallow `pandas<2`? cc @ocefpaf ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7716/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1642317716,I_kwDOAMm_X85h48eU,7685,Add welcome bot?,2448579,closed,0,,,6,2023-03-27T15:24:25Z,2023-04-06T01:55:55Z,2023-04-06T01:55:55Z,MEMBER,,,,"### Is your feature request related to a problem? Given all the outreachy interest (and perhaps just in general) it may be nice to enable a [welcome bot](https://github.com/apps/welcome) like on the Jupyter repos ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7685/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1409811164,I_kwDOAMm_X85UCALc,7162,copy of custom index does not align with original,2448579,closed,0,,,7,2022-10-14T20:17:22Z,2023-03-24T20:37:13Z,2023-03-24T20:37:12Z,MEMBER,,,,"### What happened? MY prototype CRSIndex is broken on the release version: https://github.com/dcherian/crsindex/blob/main/crsindex.ipynb under heading ""BROKEN: Successfully align with a copy of itself"" The cell's code is : ``` copy = newds.copy(deep=True) xr.align(copy, newds) ``` which should always work. @headtr1ck is https://github.com/pydata/xarray/pull/7140 to blame? ### Environment
INSTALLED VERSIONS ------------------ commit: None python: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:43:44) [Clang 13.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 2022.10.0 pandas: 1.5.0 numpy: 1.23.3 scipy: 1.9.1 netCDF4: 1.6.0 pydap: None h5netcdf: 1.0.2 h5py: 3.7.0 Nio: None zarr: 2.13.3 cftime: 1.6.2 nc_time_axis: 1.4.1 PseudoNetCDF: 3.2.2 rasterio: 1.3.2 cfgrib: 0.9.10.2 iris: 3.3.1 bottleneck: 1.3.5 dask: 2022.9.2 distributed: 2022.9.2 matplotlib: 3.6.1 cartopy: 0.21.0 seaborn: 0.12.0 numbagg: 0.2.1 fsspec: 2022.8.2 cupy: None pint: 0.19.2 sparse: 0.13.0 flox: 0.6.0 numpy_groupies: 0.9.19 setuptools: 65.5.0 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: 8.5.0 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 984555353,MDU6SXNzdWU5ODQ1NTUzNTM=,5754,Variable.stack constructs extremely large chunks,2448579,closed,0,,,6,2021-09-01T03:08:02Z,2023-03-22T14:51:44Z,2021-12-14T17:31:45Z,MEMBER,,,,"**Minimal Complete Verifiable Example**: Here's a small array with too-small chunk sizes just as an example ```python # Put your MCVE code here import dask.array import xarray as xr var = xr.Variable((""x"", ""y"", ""z""), dask.array.random.random((4, 18483, 1000), chunks=(1, 183, -1))) ``` Now stack two dimensions, this is a 100x increase in chunk size (in my actual code, 85MB chunks become 8.5GB chunks =) ) ``` var.stack(new=(""x"", ""y"")) ``` But calling `reshape` on the dask array preserves the original chunk size ``` var.data.reshape((4*18483, -1)) ``` ## Solution Ah, found it , we transpose then reshape in `Variable_stack_once`. https://github.com/pydata/xarray/blob/f915515d610b4471888fa44dfb00dbae3fd22349/xarray/core/variable.py#L1521-L1527 Writing those steps with pure dask yields the same 100x increase in chunksize ``` python var.data.transpose([2, 0, 1]).reshape((-1, 4*18483)) ``` **Anything else we need to know?**: **Environment**:
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.8.6 | packaged by conda-forge | (default, Jan 25 2021, 23:21:18) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1127.18.2.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.19.0 pandas: 1.3.1 numpy: 1.21.1 scipy: 1.5.3 netCDF4: 1.5.6 pydap: installed h5netcdf: 0.11.0 h5py: 3.3.0 Nio: None zarr: 2.8.3 cftime: 1.5.0 nc_time_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: 3.0.4 bottleneck: 1.3.2 dask: 2021.07.2 distributed: 2021.07.2 matplotlib: 3.4.2 cartopy: 0.19.0.post1 seaborn: 0.11.1 numbagg: None pint: 0.17 setuptools: 49.6.0.post20210108 pip: 21.2.2 conda: 4.10.3 pytest: 6.2.4 IPython: 7.26.0 sphinx: 4.1.2
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5754/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1530966360,I_kwDOAMm_X85bQLFY,7434,RTD failure on main,2448579,closed,0,,,2,2023-01-12T15:57:55Z,2023-01-13T17:38:00Z,2023-01-13T17:38:00Z,MEMBER,,,,"### What happened? [logs](https://readthedocs.org/projects/xray/builds/19149017/) ``` sphinx.errors.SphinxParallelError: RuntimeError: Non Expected exception in `/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/7433/doc/user-guide/interpolation.rst` line 331 ``` This seems real ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7434/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1284094480,I_kwDOAMm_X85MiboQ,6722,Avoid loading any data for reprs,2448579,closed,0,,,5,2022-06-24T19:04:30Z,2022-10-28T16:23:20Z,2022-10-28T16:23:20Z,MEMBER,,,,"### What happened? For ""small"" datasets, we load in to memory when displaying the repr. For cloud backed datasets with large number of ""small"" variables, this can use a lot of time sequentially loading O(100) variables just for a repr. https://github.com/pydata/xarray/blob/6c8db5ed005e000b35ad8b6ea9080105e608e976/xarray/core/formatting.py#L548-L549 ### What did you expect to happen? Fast reprs! ### Minimal Complete Verifiable Example This dataset has 48 ""small"" variables ```Python import xarray as xr dc1 = xr.open_dataset('s3://its-live-data/datacubes/v02/N40E080/ITS_LIVE_vel_EPSG32645_G0120_X250000_Y4750000.zarr', engine= 'zarr', storage_options = {'anon':True}) dc1._repr_html_() ``` On `2022.03.0` this repr takes 36.4s If I comment the `array.size` condition I get 6μs. ### MVCE confirmation - [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [x] Complete example — the example is self-contained, including all data and the text of any traceback. - [x] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [x] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
INSTALLED VERSIONS ------------------ commit: None python: 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.2 distributed: None matplotlib: 3.5.2 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.5.0 cupy: None pint: None sparse: None setuptools: 62.3.2 pip: 22.1.2 conda: None pytest: None IPython: 8.4.0 sphinx: 4.5.0
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6722/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1309839509,I_kwDOAMm_X85OEpCV,6810,Convert upstream-dev CI scripts to github Action,2448579,closed,0,,,2,2022-07-19T17:32:15Z,2022-10-26T09:12:43Z,2022-10-26T09:12:43Z,MEMBER,,,,"### Is your feature request related to a problem? No. ### Describe the solution you'd like If possible, I think it'd be nice to move a lot of the upstream-dev CI scripting to its own github action like ""ci-trigger"". This will make it easier to use in other projects (like those under xarray-contrib). I'd like to use it for flox, cf-xarray. ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6810/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1404926762,I_kwDOAMm_X85TvXsq,7154,nightly failure with h5netcdf indexing,2448579,closed,0,,,8,2022-10-11T16:32:33Z,2022-10-12T14:11:04Z,2022-10-12T14:11:04Z,MEMBER,,,,"### What happened? From upstream-dev CI: [Workflow Run URL](https://github.com/pydata/xarray/actions/runs/3223203335)
Python 3.10 Test Summary ``` xarray/tests/test_backends.py::TestH5NetCDFData::test_orthogonal_indexing: AssertionError: Left and right Dataset objects are not identical Differing coordinates: L numbers (dim3) int64 0 1 2 0 0 R numbers (dim3) int64 ... L * dim3 (dim3) cc @benbovy @kmuehlbauer ### Environment
INSTALLED VERSIONS ------------------ commit: 8eea8bb67bad0b5ac367c082125dd2b2519d4f52 python: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:35:26) [GCC 10.4.0] python-bits: 64 OS: Linux OS-release: 5.15.0-1020-azure machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 2022.9.1.dev12+g8eea8bb6 pandas: 1.6.0.dev0+297.g55dc32437e numpy: 1.24.0.dev0+896.g5ecaf36cd scipy: 1.10.0.dev0+2012.5be8bc4 netCDF4: 1.6.0 pydap: installed h5netcdf: 1.1.0.dev5+g1168b4f h5py: 3.7.0 Nio: None zarr: 2.13.4.dev1 cftime: 1.6.2 nc_time_axis: 1.3.1.dev34+g0999938 PseudoNetCDF: 3.2.2 rasterio: 1.4dev cfgrib: 0.9.10.2 iris: 3.3.1 bottleneck: 1.3.5 dask: 2022.9.2+17.g5ba240b9 distributed: 2022.9.2+19.g07e22593 matplotlib: 3.7.0.dev320+g834c89c512 cartopy: 0.21.0 seaborn: 0.12.0 numbagg: None fsspec: 2022.8.2+14.g3969aaf cupy: None pint: 0.19.3.dev87+g052a920 sparse: None flox: 0.5.11.dev3+g031979d numpy_groupies: 0.9.19 setuptools: 65.4.1 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: None sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7154/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1333514579,I_kwDOAMm_X85Pe9FT,6902,Flox based groupby operations don't support `dtype` in mean method,2448579,closed,0,,,3,2022-08-09T16:38:25Z,2022-10-11T17:45:27Z,2022-10-11T17:45:27Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/6901
Originally posted by **tasansal** August 9, 2022 We have been using the new groupby logic with Flox and numpy_groupies; however, when we run the following, the dtype is not recognized as a valid argument. This breaks API compatibility for cases where you may not have the acceleration libraries installed. Not sure if this has to be upstream in In addition to base Xarray we have the following extras installed: Flox numpy_groupies Bottleneck We do this because our data is `float32` but we want the accumulator in mean to be `float64` for accuracy. One solution is to cast the variable to float64 before mean, which may cause a copy and spike in memory usage. When Flox and numpy_groupies are not installed, it works as expected. We are working with multi-dimensional time-series of weather forecast models. ```python da = xr.load_mfdataset(...) da.groupby(""time.month"").mean(dtype='float64').compute() ``` Here is the end of the traceback and it appears it is on Flox. ```shell File ""/home/altay_sansal_tgs_com/miniconda3/envs/wind-data-mos/lib/python3.10/site-packages/flox/core.py"", line 786, in _aggregate return _finalize_results(results, agg, axis, expected_groups, fill_value, reindex) File ""/home/altay_sansal_tgs_com/miniconda3/envs/wind-data-mos/lib/python3.10/site-packages/flox/core.py"", line 747, in _finalize_results finalized[agg.name] = agg.finalize(*squeezed[""intermediates""], **agg.finalize_kwargs) TypeError: () got an unexpected keyword argument 'dtype' ``` What is the best way to handle this, maybe fix it in Flox?
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6902/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1382753751,I_kwDOAMm_X85SayXX,7069,release?,2448579,closed,0,,,5,2022-09-22T17:00:58Z,2022-10-01T18:25:13Z,2022-10-01T18:25:13Z,MEMBER,,,,"### What is your issue? It's been 3 months since our last release. We still have quite a few [regressions](https://github.com/pydata/xarray/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Aregression) from the last release but @benbovy does have open PRs for a number of them. However, we do have some nice bugfixes and other commits in the mean time. I propose we issue a new release, perhaps after @benbovy merges the PRs he thinks are ready. I'll be out of town for the next few days, so if someone else could volunteer to be release manager that would be great!","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7069/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 626591460,MDU6SXNzdWU2MjY1OTE0NjA=,4107,renaming Variable to a dimension name does not convert to IndexVariable,2448579,closed,0,4160723,,0,2020-05-28T15:11:49Z,2022-09-27T09:33:42Z,2022-09-27T09:33:42Z,MEMBER,,,," Seen in #4103 #### MCVE Code Sample ```python from xarray.tests import assert_identical coord_1 = xr.DataArray([1, 2], dims=[""coord_1""], attrs={""attrs"": True}) da = xr.DataArray([1, 0], [coord_1]) obj = da.reset_index(""coord_1"").rename({""coord_1_"": ""coord_1""}) assert_identical(da, obj) ``` #### Expected Output #### Problem Description ``` AssertionErrorTraceback (most recent call last) in ----> 1 assert_identical(da, obj) ~/work/python/xarray/xarray/tests/__init__.py in assert_identical(a, b) 160 xarray.testing.assert_identical(a, b) 161 xarray.testing._assert_internal_invariants(a) --> 162 xarray.testing._assert_internal_invariants(b) 163 164 ~/work/python/xarray/xarray/testing.py in _assert_internal_invariants(xarray_obj) 265 _assert_variable_invariants(xarray_obj) 266 elif isinstance(xarray_obj, DataArray): --> 267 _assert_dataarray_invariants(xarray_obj) 268 elif isinstance(xarray_obj, Dataset): 269 _assert_dataset_invariants(xarray_obj) ~/work/python/xarray/xarray/testing.py in _assert_dataarray_invariants(da) 210 assert all( 211 isinstance(v, IndexVariable) for (k, v) in da._coords.items() if v.dims == (k,) --> 212 ), {k: type(v) for k, v in da._coords.items()} 213 for k, v in da._coords.items(): 214 _assert_variable_invariants(v, k) AssertionError: {'coord_1': } ``` #### Versions
Output of xr.show_versions()
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1315480779,I_kwDOAMm_X85OaKTL,6817,wrong mean of complex values,2448579,closed,0,,,1,2022-07-22T23:09:47Z,2022-07-23T02:03:11Z,2022-07-23T02:03:11Z,MEMBER,,,,"### What happened? Seen in #4972 ``` python import xarray as xr import numpy as np array = np.array([0. +0.j, 0.+np.nan * 1j], dtype=np.complex64) var = xr.Variable(""x"", array) print(var.mean().data) print(array.mean()) ``` ``` 0j (nan+nanj) ``` ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6817/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1298145215,I_kwDOAMm_X85NYB-_,6763,Map_blocks should raise nice error if provided template has no dask arrays,2448579,closed,0,,,3,2022-07-07T21:58:06Z,2022-07-14T17:42:26Z,2022-07-14T17:42:26Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/6762
Originally posted by **tlsw231** July 7, 2022 I am trying to use `map_blocks` to: ingest a multi-dimensional array as input, reduce along one dimension and add extra dimensions to the output. Is this possible? I am attaching a simple MRE below that gives me an `zip argument #2 must support iteration` error. Any pointers on what I might be doing wrong? [My real example is a 3d-dataset with `(time,lat,lon)` dimensions and I am trying to reduce along `time` while adding two new dimensions to the output. I tried so many things and got so many errors, including the one in the title, that I thought it is better to first understand how `map_blocks` works!] ``` # The goal is to feed in a 2d array, reduce along one dimension and add two new dimensions to the output. chunks={} dummy = xr.DataArray(data=np.random.random([8,100]),dims=['dim1','dim2']).chunk(chunks) def some_func(func): dims=func.dims n1 = len(func[func.dims[1]]) # This is 'dim2', we will average along 'dim1' below in the for loop newdim1 = 2; newdim2 = 5; output = xr.DataArray(np.nan*np.ones([n1,newdim1,newdim2]),dims=[dims[1],'new1','new2']) for n in range(n1): fmean = func.isel(dim2=n).mean(dims[0]).compute() for i in range(newdim1): for j in range(newdim2): output[n,i,j] = fmean return output #out = some_func(dummy) # This works template=xr.DataArray(np.nan*np.ones([len(dummy.dim2),2,5]), dims=['dim2','new1','new2']) out = xr.map_blocks(some_func,dummy,template=template).compute() # gives me the error message in the title ``` [Edit: Fixed a typo in the `n1 = len(func[func.dims[1]])` line, of course getting the same error.]
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6763/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1289174987,I_kwDOAMm_X85M1z_L,6739,"""center"" kwarg ignored when manually iterating over DataArrayRolling",2448579,closed,0,,,0,2022-06-29T19:07:07Z,2022-07-14T17:41:01Z,2022-07-14T17:41:01Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/6738
Originally posted by **ckingdon95** June 29, 2022 Hello, I am trying to manually iterate over a DataArrayRolling object, as described [here ](https://docs.xarray.dev/en/stable/user-guide/computation.html#rolling-window-operations)in the documentation. I am confused why the following two code chunks do not produce the same sequence of values. I would like to be able to manually iterate over a DataArrayRolling object, and still be given center-justified windows. Is there a way to do this? ```python import xarray as xr import numpy as np my_data = xr.DataArray(np.arange(1,10), dims=""x"") # Option 1: take a center-justified rolling average result1 = my_data.rolling(x=3, center=True).mean().values result1 ``` This returns the following values, as expected: ``` array([nan, 2., 3., 4., 5., 6., 7., 8., nan]) ``` Whereas when I do it manually, it is not equivalent: ```python # Option 2: try to manually iterate, but the result is not centered my_data_rolling = my_data.rolling(x=3, center=True) result2 = [window.mean().values.item() for label, window in my_data_rolling] result2 ``` This returns ``` [nan, nan, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0] ``` Is this an issue with the window iterator? If it is not an issue, then is there a way for me to get the center-justified windows in the manual iteration?
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1290524064,I_kwDOAMm_X85M69Wg,6741,some private imports broken on main,2448579,closed,0,,,6,2022-06-30T18:59:28Z,2022-07-06T03:06:31Z,2022-07-06T03:06:31Z,MEMBER,,,,"### What happened? Seen over in [cf_xarray](https://github.com/xarray-contrib/cf-xarray/pull/343) Using `xr.core.resample.Resample` worked prior to https://github.com/pydata/xarray/pull/6702. Now we need to use `from xarray.core.resample import Resample` I don't know if this is something that needs to be fixed or only worked coincidentally earlier. But I thought it was worth discussing prior to release. Thanks to @aulemahal for spotting ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6741/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 968977385,MDU6SXNzdWU5Njg5NzczODU=,5699,describe options in documentation,2448579,closed,0,,,0,2021-08-12T14:48:00Z,2022-06-25T20:01:07Z,2022-06-25T20:01:07Z,MEMBER,,,,"I think we only describe available options in the API reference for `xr.set_options` It'd be nice to add a ""Configuring Xarray"" section in the User Guide. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5699/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1178907807,I_kwDOAMm_X85GRLSf,6407,Add backend tutorial material,2448579,closed,0,,,0,2022-03-24T03:44:22Z,2022-06-23T01:51:44Z,2022-06-23T01:51:44Z,MEMBER,,,,"### What is your issue? @aurghs developed some nice backend tutorial material for the Dask Summit: https://github.com/aurghs/xarray-backend-tutorial It'd be nice to add it either to our main documentation or to https://github.com/xarray-contrib/xarray-tutorial.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1258338848,I_kwDOAMm_X85LALog,6659,Publish nightly releases to TestPyPI,2448579,closed,0,,,6,2022-06-02T15:21:24Z,2022-06-07T08:37:02Z,2022-06-06T22:33:15Z,MEMBER,,,,"### Is your feature request related to a problem? From @keewis in #6645 > if anyone can figure out how to create PEP440 (and thus PyPI) compatible development versions I think we can have a CI publish every commit on main to TestPyPI. ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6659/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1238783899,I_kwDOAMm_X85J1leb,6616,flox breaks multiindex groupby,2448579,closed,0,,,0,2022-05-17T15:05:00Z,2022-05-17T16:11:18Z,2022-05-17T16:11:18Z,MEMBER,,,,"### What happened? From @malmans2 ``` python import numpy as np import xarray as xr ds = xr.Dataset( dict(a=((""z"",), np.ones(10))), coords=dict(b=((""z""), np.arange(2).repeat(5)), c=((""z""), np.arange(5).repeat(2))), ).set_index(bc=[""b"", ""c""]) grouped = ds.groupby(""bc"") with xr.set_options(use_flox=False): grouped.sum() # OK with xr.set_options(use_flox=True): grouped.sum() # Error ``` ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output ```Python ctests/test_xarray.py:329: in test_multi_index_groupby_sum actual = xarray_reduce(ds, ""bc"", func=""sum"") flox/xarray.py:374: in xarray_reduce actual[k] = v.expand_dims(missing_group_dims) ../xarray/xarray/core/dataset.py:1427: in __setitem__ self.update({key: value}) ../xarray/xarray/core/dataset.py:4432: in update merge_result = dataset_update_method(self, other) ../xarray/xarray/core/merge.py:1070: in dataset_update_method return merge_core( ../xarray/xarray/core/merge.py:722: in merge_core aligned = deep_align( ../xarray/xarray/core/alignment.py:824: in deep_align aligned = align( ../xarray/xarray/core/alignment.py:761: in align aligner.align() ../xarray/xarray/core/alignment.py:550: in align self.assert_unindexed_dim_sizes_equal() ../xarray/xarray/core/alignment.py:450: in assert_unindexed_dim_sizes_equal raise ValueError( E ValueError: cannot reindex or align along dimension 'bc' because of conflicting dimension sizes: {10, 6} (note: an index is found along that dimension with size=10) ______________________________ test_multi_index_groupby_sum[numpy] _______________________________ tests/test_xarray.py:329: in test_multi_index_groupby_sum actual = xarray_reduce(ds, ""bc"", func=""sum"") flox/xarray.py:374: in xarray_reduce actual[k] = v.expand_dims(missing_group_dims) ../xarray/xarray/core/dataset.py:1427: in __setitem__ self.update({key: value}) ../xarray/xarray/core/dataset.py:4432: in update merge_result = dataset_update_method(self, other) ../xarray/xarray/core/merge.py:1070: in dataset_update_method return merge_core( ../xarray/xarray/core/merge.py:722: in merge_core aligned = deep_align( ../xarray/xarray/core/alignment.py:824: in deep_align aligned = align( ../xarray/xarray/core/alignment.py:761: in align aligner.align() ../xarray/xarray/core/alignment.py:550: in align self.assert_unindexed_dim_sizes_equal() ../xarray/xarray/core/alignment.py:450: in assert_unindexed_dim_sizes_equal raise ValueError( E ValueError: cannot reindex or align along dimension 'bc' because of conflicting dimension sizes: {10, 6} (note: an index is found along that dimension with size=10) ``` ``` ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6616/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1235494254,I_kwDOAMm_X85JpCVu,6606,Fix benchmark CI,2448579,closed,0,,,0,2022-05-13T17:18:32Z,2022-05-14T23:06:44Z,2022-05-14T23:06:44Z,MEMBER,,,,"### What is your issue? It's failing during setup: https://github.com/pydata/xarray/runs/6424624397?check_suite_focus=true ``` · Discovering benchmarks ·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Building dd20d07f for conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/bin/python -mpip wheel --no-deps --no-index -w /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/asv-build-cache/dd20d07f4057a9e29222ca132c36cbaaf3fbb242 /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project (exit status 1) STDOUT --------> Processing /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project STDERR --------> ERROR: Some build dependencies for file:///home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project are missing: 'setuptools_scm[toml]>=3.4', 'setuptools_scm_git_archive'. ·· Failed: trying different commit/environment ·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Building c34ef8a6 for conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/bin/python -mpip wheel --no-deps --no-index -w /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/asv-build-cache/c34ef8a60227720724e90aa11a6266c0026a812a /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project (exit status 1) STDOUT --------> Processing /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project STDERR --------> ERROR: Some build dependencies for file:///home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project are missing: 'setuptools_scm[toml]>=3.4', 'setuptools_scm_git_archive'. ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1180334986,I_kwDOAMm_X85GWnuK,6411,Better dask support in polyval,2448579,closed,0,,,0,2022-03-25T04:35:48Z,2022-05-05T20:17:07Z,2022-05-05T20:17:07Z,MEMBER,,,,"### Is your feature request related to a problem? polyval does not handle dask inputs well. ```python nt = 8772 // 4 ny = 489 nx = 655 # chunks like the data is stored on disk # small in time, big in space # because the chunk sizes are -1 along lat, lon; # reshaping this array to (time, latlon) prior to fitting is pretty cheap chunks = (8, -1, -1) da = xr.DataArray( dask.array.random.random((nt, ny, nx), chunks=chunks), dims=(""ocean_time"", ""eta_rho"", ""xi_rho""), ) dim = ""ocean_time"" deg = 1 p = da.polyfit(dim=""ocean_time"", deg=1, skipna=False) # create a chunked version of the ""ocean_time"" dimension chunked_dim = xr.DataArray( dask.array.from_array(da[dim].data, chunks=da.chunksizes[dim]), dims=dim, name=dim ) xr.polyval(chunked_dim, p.polyfit_coefficients) ``` ### Describe the solution you'd like Here's a partial solution. It does not handle datetime inputs (polyval handles this using `get_clean_interp_index` which computes dask inputs). But I've replaced the call to `np.vander` and used `xr.dot`. ```python def polyval(coord, coeffs, degree_dim=""degree""): x = coord.data deg_coord = coeffs[degree_dim] N = int(deg_coord.max()) + 1 lhs = xr.DataArray( np.stack([x ** (N - 1 - i) for i in range(N)], axis=1), dims=(coord.name, degree_dim), coords={coord.name: coord, degree_dim: np.arange(deg_coord.max() + 1)[::-1]}, ) return xr.dot(lhs, coeffs, dims=degree_dim) polyval(chunked_dim, p.polyfit_coefficients) ``` This looks like what I expected cc @aulemahal ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1207159549,I_kwDOAMm_X85H88r9,6497,restrict stale bot,2448579,closed,0,,,1,2022-04-18T15:25:56Z,2022-04-18T16:11:11Z,2022-04-18T16:11:11Z,MEMBER,,,,"### What is your issue? We have some stale issue but not that many. Can we restrict the bot to only issues that are untagged, or tagged as ""usage question"" or are not assigned to a ""project"" instead? This might reduce a lot of the noise.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6497/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1188406993,I_kwDOAMm_X85G1abR,6430,Bug in broadcasting with multi-indexes,2448579,closed,0,,,1,2022-03-31T17:25:57Z,2022-04-13T14:49:23Z,2022-04-13T14:49:23Z,MEMBER,,,,"### What happened? ``` python import numpy as np import xarray as xr ds = xr.Dataset( {""foo"": ((""x"", ""y"", ""z""), np.ones((3, 4, 2)))}, {""x"": [""a"", ""b"", ""c""], ""y"": [1, 2, 3, 4]}, ) expected = ds.sum(""z"") stacked = ds.stack(space=[""x"", ""y""]) broadcasted, _ = xr.broadcast(stacked, stacked.space) stacked.sum(""z"").unstack(""space"") # works broadcasted.sum(""z"").unstack(""space"") # error ``` ``` --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Input In [13], in 10 broadcasted, _ = xr.broadcast(stacked, stacked.space) 11 stacked.sum(""z"").unstack(""space"") ---> 12 broadcasted.sum(""z"").unstack(""space"") File ~/work/python/xarray/xarray/core/dataset.py:4332, in Dataset.unstack(self, dim, fill_value, sparse) 4330 non_multi_dims = set(dims) - set(stacked_indexes) 4331 if non_multi_dims: -> 4332 raise ValueError( 4333 ""cannot unstack dimensions that do not "" 4334 f""have exactly one multi-index: {tuple(non_multi_dims)}"" 4335 ) 4337 result = self.copy(deep=False) 4339 # we want to avoid allocating an object-dtype ndarray for a MultiIndex, 4340 # so we can't just access self.variables[v].data for every variable. 4341 # We only check the non-index variables. 4342 # https://github.com/pydata/xarray/issues/5902 ValueError: cannot unstack dimensions that do not have exactly one multi-index: ('space',) ``` ### What did you expect to happen? This should work. ### Minimal Complete Verifiable Example _No response_ ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment xarray main after the flexible indexes refactor","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6430/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1193704369,I_kwDOAMm_X85HJnux,6444,xr.where with scalar as second argument fails with keep_attrs=True,2448579,closed,0,,,1,2022-04-05T20:51:18Z,2022-04-12T02:12:39Z,2022-04-12T02:12:39Z,MEMBER,,,,"### What happened? ``` python import xarray as xr xr.where(xr.DataArray([1, 2, 3]) > 0, 1, 0) ``` fails with ``` 1809 if keep_attrs is True: 1810 # keep the attributes of x, the second parameter, by default to 1811 # be consistent with the `where` method of `DataArray` and `Dataset` -> 1812 keep_attrs = lambda attrs, context: attrs[1] 1814 # alignment for three arguments is complicated, so don't support it yet 1815 return apply_ufunc( 1816 duck_array_ops.where, 1817 cond, (...) 1823 keep_attrs=keep_attrs, 1824 ) IndexError: list index out of range ``` The workaround is to pass `keep_attrs=False` ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment xarray 2022.3.0","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6444/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 528168017,MDU6SXNzdWU1MjgxNjgwMTc=,3573,rasterio test failure,2448579,closed,0,,,1,2019-11-25T15:40:19Z,2022-04-09T01:17:32Z,2022-04-09T01:17:32Z,MEMBER,,,,"version ``` rasterio 1.1.1 py36h900e953_0 conda-forge ``` ``` =================================== FAILURES =================================== ________________________ TestRasterio.test_rasterio_vrt ________________________ self = def test_rasterio_vrt(self): import rasterio # tmp_file default crs is UTM: CRS({'init': 'epsg:32618'} with create_tmp_geotiff() as (tmp_file, expected): with rasterio.open(tmp_file) as src: with rasterio.vrt.WarpedVRT(src, crs=""epsg:4326"") as vrt: expected_shape = (vrt.width, vrt.height) expected_crs = vrt.crs expected_res = vrt.res # Value of single pixel in center of image lon, lat = vrt.xy(vrt.width // 2, vrt.height // 2) > expected_val = next(vrt.sample([(lon, lat)])) xarray/tests/test_backends.py:3966: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/share/miniconda/envs/xarray-tests/lib/python3.6/site-packages/rasterio/sample.py:43: in sample_gen data = read(indexes, window=window, masked=masked, boundless=True) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ??? E ValueError: WarpedVRT does not permit boundless reads rasterio/_warp.pyx:978: ValueError ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3573/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1189140909,I_kwDOAMm_X85G4Nmt,6434,concat along dim with mix of scalar coordinate and array coordinates is not right,2448579,closed,0,,,3,2022-04-01T02:29:16Z,2022-04-06T01:19:47Z,2022-04-06T01:19:47Z,MEMBER,,,,"### What happened? Really hard to describe in words =) ``` concat = xr.concat([da.isel(time=0), da.isel(time=[1])], dim=""time"") xr.align(concat, da, dim=""time"") ``` fails when `concat` and `da` should be identical. This is causing failures in cf-xarray:https://github.com/xarray-contrib/cf-xarray/issues/319 cc @benbovy ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example ```Python import numpy as np import xarray as xr time = xr.DataArray( np.array( [""2013-01-01T00:00:00.000000000"", ""2013-01-01T06:00:00.000000000""], dtype=""datetime64[ns]"", ), dims=""time"", name=""time"", ) da = time concat = xr.concat([da.isel(time=0), da.isel(time=[1])], dim=""time"") xr.align(da, concat, join=""exact"") # works da = xr.DataArray(np.ones(time.shape), dims=""time"", coords={""time"": time}) concat = xr.concat([da.isel(time=0), da.isel(time=[1])], dim=""time"") xr.align(da, concat, join=""exact"") ``` ### Relevant log output ``` --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Input In [27], in 17 da = xr.DataArray(np.ones(time.shape), dims=""time"", coords={""time"": time}) 18 concat = xr.concat([da.isel(time=0), da.isel(time=[1])], dim=""time"") ---> 19 xr.align(da, concat, join=""exact"") File ~/work/python/xarray/xarray/core/alignment.py:761, in align(join, copy, indexes, exclude, fill_value, *objects) 566 """""" 567 Given any number of Dataset and/or DataArray objects, returns new 568 objects with aligned indexes and dimension sizes. (...) 751 752 """""" 753 aligner = Aligner( 754 objects, 755 join=join, (...) 759 fill_value=fill_value, 760 ) --> 761 aligner.align() 762 return aligner.results File ~/work/python/xarray/xarray/core/alignment.py:549, in Aligner.align(self) 547 self.find_matching_unindexed_dims() 548 self.assert_no_index_conflict() --> 549 self.align_indexes() 550 self.assert_unindexed_dim_sizes_equal() 552 if self.join == ""override"": File ~/work/python/xarray/xarray/core/alignment.py:395, in Aligner.align_indexes(self) 393 if need_reindex: 394 if self.join == ""exact"": --> 395 raise ValueError( 396 ""cannot align objects with join='exact' where "" 397 ""index/labels/sizes are not equal along "" 398 ""these coordinates (dimensions): "" 399 + "", "".join(f""{name!r} {dims!r}"" for name, dims in key[0]) 400 ) 401 joiner = self._get_index_joiner(index_cls) 402 joined_index = joiner(matching_indexes) ValueError: cannot align objects with join='exact' where index/labels/sizes are not equal along these coordinates (dimensions): 'time' ('time',) ``` ### Anything else we need to know? _No response_ ### Environment xarray main","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6434/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1001197796,I_kwDOAMm_X847rRDk,5804,vectorized groupby binary ops,2448579,closed,0,,,1,2021-09-20T17:04:47Z,2022-03-29T07:11:28Z,2022-03-29T07:11:28Z,MEMBER,,,,"By switching to `numpy_groupies` we are vectorizing our groupby reductions. I think we can do the same for groupby's binary ops. Here's an example array ``` python import numpy as np import xarray as xr %load_ext memory_profiler N = 4 * 2000 da = xr.DataArray( np.random.random((N, N)), dims=(""x"", ""y""), coords={""labels"": (""x"", np.repeat([""a"", ""b"", ""c"", ""d"", ""e"", ""f"", ""g"", ""h""], repeats=N//8))}, ) ``` Consider this ""anomaly"" calculation, anomaly defined relative to the group mean ``` python def anom_current(da): grouped = da.groupby(""labels"") mean = grouped.mean() anom = grouped - mean return anom ``` With this approach, we loop over each group and apply the binary operation: https://github.com/pydata/xarray/blob/a1635d324753588e353e4e747f6058936fa8cf1e/xarray/core/computation.py#L502-L525 This saves some memory, but becomes slow for large number of groups. We could instead do ``` def anom_vectorized(da): mean = da.groupby(""labels"").mean() mean_expanded = mean.sel(labels=da.labels) anom = da - mean_expanded return anom ``` Now we are faster, but construct an extra array as big as the original array (I think this is an OK tradeoff). ``` %timeit anom_current(da) # 1.4 s ± 20.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) %timeit anom_vectorized(da) # 937 ms ± 5.26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ``` ------ (I haven't experimented with dask yet, so the following is just a theory). I think the real benefit comes with dask. Depending on where the groups are located relative to chunking, we could end up creating a lot of tiny chunks by splitting up existing chunks. With the vectorized approach we can do better. Ideally we would reindex the ""mean"" dask array with a numpy-array-of-repeated-ints such that the chunking of `mean_expanded` exactly matches the chunking of `da` along the grouped dimension. ~In practice, [dask.array.take](https://docs.dask.org/en/latest/_modules/dask/array/routines.html#take) doesn't allow specifying ""output chunks"" so we'd end up chunking ""mean_expanded"" based on dask's automatic heuristics, and then rechunking again for the binary operation.~ Thoughts? cc @rabernat","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5804/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1178949620,I_kwDOAMm_X85GRVf0,6408,backwards incompatible changes in reductions,2448579,closed,0,,,2,2022-03-24T04:11:00Z,2022-03-26T08:44:43Z,2022-03-26T08:44:43Z,MEMBER,,,,"### What is your issue? I merged #5950 but forgot that it included some backward-incompatible changes (Sorry! this came up in https://github.com/pydata/xarray/pull/6403 thanks to @mathause for spotting.) 1. Arguments like `keep_attrs`, `axis` are now keyword-only. 2. Some reductions had the 3rd position arg as `keep_attrs` and in other cases it was `axis`. These have been standardized now, and only `dim` is accepted without kwarg-name. @pydata/xarray Should we add a deprecation cycle (https://github.com/pydata/xarray/issues/5531)? Point (2) above will make it a little messy. At the very least we should add a deprecation notice before releasing. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6408/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1174177534,I_kwDOAMm_X85F_Ib-,6381,vectorized indexing with DataArray should not preserve IndexVariable,2448579,closed,0,,,1,2022-03-19T05:08:39Z,2022-03-21T04:47:47Z,2022-03-21T04:47:47Z,MEMBER,,,,"### What happened? After vectorized indexing a DataArray with dim `x`by a DataArray `z`, we get a DataArray with dim `z` and `x` as non-dim coordinate. But `x` is still an IndexVariable, not a normal variable. ### What did you expect to happen? `x` should be a normal variable. ### Minimal Complete Verifiable Example ```python import xarray as xr xr.set_options(display_style=""text"") da = xr.DataArray([1, 2, 3], dims=""x"", coords={""x"": [0, 1, 2]}) idxr = xr.DataArray([1], dims=""z"", name=""x"", coords={""z"": (""z"", [""a""])}) da.sel(x=idxr) ``` ``` array([2]) Coordinates: x (z) int64 1 * z (z) array([1]) ``` ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment xarray main but this bug was present prior to the explicit indexes refactor.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6381/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1171932478,I_kwDOAMm_X85F2kU-,6373,Zarr backend should avoid checking for invalid encodings,2448579,closed,0,,,3,2022-03-17T04:55:35Z,2022-03-18T10:06:01Z,2022-03-18T04:19:48Z,MEMBER,,,,"### What is your issue? The zarr backend has a list of ""valid"" encodings that needs to be updated any time zarr adds something new (e.g. https://github.com/pydata/xarray/pull/6348) https://github.com/pydata/xarray/blob/53172cb1e03a98759faf77ef48efaa64676ad24a/xarray/backends/zarr.py#L215-L234 Can we get rid of this? I don't know the backends code well, but won't all our encoding parameters have been removed by this stage? The `raise_on_invalid` kwarg suggests so. @tomwhite points out that zarr will raise a warning: ``` python >>> zarr.create((1,), blah=1) /Users/tom/miniconda/envs/sgkit-3.8/lib/python3.8/site-packages/zarr/creation.py:221: UserWarning: ignoring keyword argument 'blah' warn('ignoring keyword argument %r' % k) ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6373/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1170533154,I_kwDOAMm_X85FxOsi,6363,failing flaky test: rasterio vrt,2448579,closed,0,,,2,2022-03-16T04:38:53Z,2022-03-17T06:25:22Z,2022-03-17T06:25:22Z,MEMBER,,,,"### What happened? This test is failing with a 404 error: https://github.com/pydata/xarray/blob/95bb9ae4233c16639682a532c14b26a3ea2728f3/xarray/tests/test_backends.py#L4778-L4802 ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment N/A","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6363/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1119738354,I_kwDOAMm_X85Cvdny,6222,test packaging & distribution,2448579,closed,0,,,4,2022-01-31T17:42:40Z,2022-02-03T15:45:17Z,2022-02-03T15:45:17Z,MEMBER,,,,"### Is your feature request related to a problem? It seems like we should have a test to make sure our dependencies are specified correctly. ### Describe the solution you'd like For instance we could add a step to the release workflow: https://github.com/pydata/xarray/blob/b09de8195a9e22dd35d1b7ed608ea15dad0806ef/.github/workflows/pypi-release.yaml#L34-L43 after `twine check` where we pip install and then try to import xarray. Alternatively we could have another test config in our regular CI to build + import. Thoughts? Is this excessive for a somewhat rare problem? ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6222/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1072473598,I_kwDOAMm_X84_7KX-,6051,"Check for just ... in stack etc, and raise with a useful error message",2448579,closed,0,,,4,2021-12-06T18:35:27Z,2022-01-03T23:05:23Z,2022-01-03T23:05:23Z,MEMBER,,,," **Is your feature request related to a problem? Please describe.** The following doesn't work ``` python import xarray as xr da = xr.DataArray([[1,2],[1,2]], dims=(""x"", ""y"")) da.stack(flat=...) ``` **Describe the solution you'd like** This could be equivalent to ``` python da.stack(flat=da.dims) ``` I *think* using `ds.dims` it should be fine for datasets too. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6051/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 767941842,MDU6SXNzdWU3Njc5NDE4NDI=,4697,deprecate open_rasterio in favour of a rioxarray entrypoint?,2448579,closed,0,,,8,2020-12-15T18:05:58Z,2021-10-02T20:38:35Z,2021-10-02T20:38:35Z,MEMBER,,,,"Now that our backends work is well underway, it seems like a good time to discuss deprecating the `open_rasterio` function and outsourcing it to `rioxarray` (which would then implement support for the new entrypoint). From @jhamman's [comment](https://github.com/pydata/xarray/issues/4142#issuecomment-642413509) > There's also the question of where the rasterio backend should live once #1970 is implemented. I think we can make a pretty strong argument that it should move out of xarray and into a 3rd party package. Perhaps rasterio itself, or, more likely something like rio-xarray (cc @snowman2). See #3166 for a demo of how this may work. > As you can see, there are a few big questions up in the air right now. Work on #1970 will begin this summer so now is a great time to game out the various options for the rasterio backend. I am in favour of doing this. I see no reason why there should be two slightly divergent versions of the same function, and (anecdotally) the `rioxarray` version seems to be more full-featured and regularly improved. @pydata/xarray What do you think? If we agree to do this (and rioxarray agrees), we could put this as a ""deliverable"" in our NASA proposal. This work would be totally relevant to that call. Related: #4142 cc @snowman2 @scottyhq @JessicaS11 @WeatherGod ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4697/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 985376146,MDU6SXNzdWU5ODUzNzYxNDY=,5758,RTD build failing,2448579,closed,0,,,6,2021-09-01T16:50:58Z,2021-09-08T09:47:17Z,2021-09-08T09:47:16Z,MEMBER,,,,"The current RTD build is failing in `plotting.rst` ``` sphinx.errors.SphinxParallelError: RuntimeError: Non Expected exception in `/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/latest/doc/user-guide/plotting.rst` line None Sphinx parallel build error: RuntimeError: Non Expected exception in `/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/latest/doc/user-guide/plotting.rst` line None [IPKernelApp] WARNING | Parent appears to have exited, shutting down. ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5758/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 968970143,MDU6SXNzdWU5Njg5NzAxNDM=,5698,Show what options are enabled.,2448579,closed,0,,,2,2021-08-12T14:43:57Z,2021-09-04T23:44:25Z,2021-09-04T23:44:25Z,MEMBER,,,,"As far as I know, there's no way to see what options are enabled. It'd be nice to add either `xr.get_options` or `xr.show_options` or `xr.options` so that users can see what is enabled. I think it would be nice to have a one-line description under each option ``` ""arithmetic_join"": ""outer"" controls DataArray/Dataset alignment in binary operations. ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5698/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 484240082,MDU6SXNzdWU0ODQyNDAwODI=,3245,sparse and other duck array issues,2448579,closed,0,,,33,2019-08-22T22:10:29Z,2021-07-21T21:42:48Z,2021-07-21T21:42:48Z,MEMBER,,,,"Issues I've run into working with @friedrichknuth at the Pangeo meeting trying to use sparse 1. `.plot()` doesn't work. We need to convert to dense before plotting 2. `.values` doesn't work. Raises an error about explicitly calling `todense()` 3. `align` doesn't seem to work.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3245/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 484270833,MDU6SXNzdWU0ODQyNzA4MzM=,3248,combine_by_coords fails with DataArrays,2448579,closed,0,,,11,2019-08-23T00:11:38Z,2021-07-02T21:34:36Z,2021-07-02T21:34:36Z,MEMBER,,,,"#### MCVE Code Sample ```python da1 = xr.DataArray([1, 2, 3], dims='x', coords={'x': [0, 1, 2]}) da2 = xr.DataArray([3, 4, 5], dims='x', coords={'x': [2, 3, 4]}) xr.combine_by_coords([da1, da2]) ``` ``` --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in 1 da1 = xr.DataArray([1, 2, 3], dims='x', coords={'x': [0, 1, 2]}) 2 da2 = xr.DataArray([3, 4, 5], dims='x', coords={'x': [2, 3, 4]}) ----> 3 xr.combine_by_coords([da1, da2]) ~/work/python/xarray/xarray/core/combine.py in combine_by_coords(datasets, compat, data_vars, coords, fill_value, join) 619 compat=compat, 620 fill_value=fill_value, --> 621 join=join, 622 ) 623 ~/work/python/xarray/xarray/core/merge.py in merge(objects, compat, join, fill_value) 588 ) 589 --> 590 obj = obj.to_dataset() if isinstance(obj, DataArray) else obj 591 dict_like_objects.append(obj) 592 ~/work/python/xarray/xarray/core/dataarray.py in to_dataset(self, dim, name) 478 return self._to_dataset_split(dim) 479 else: --> 480 return self._to_dataset_whole(name) 481 482 @property ~/work/python/xarray/xarray/core/dataarray.py in _to_dataset_whole(self, name, shallow_copy) 426 if name is None: 427 raise ValueError( --> 428 ""unable to convert unnamed DataArray to a "" 429 ""Dataset without providing an explicit name"" 430 ) ValueError: unable to convert unnamed DataArray to a Dataset without providing an explicit name ``` To get what I want, I need ```xr.combine_by_coords([da1.to_dataset(name='a'), da2.to_dataset(name='a')]).a``` I think the issue is that the code uses `to_dataset` instead of `_to_temp_dataset` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3248/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 916318941,MDU6SXNzdWU5MTYzMTg5NDE=,5454,Add coarsen.construct,2448579,closed,0,,,2,2021-06-09T15:08:35Z,2021-06-24T16:55:25Z,2021-06-24T16:55:25Z,MEMBER,,,,"Like `Rolling.construct` this would create a new reshaped array with extra dimensions that would usually be reduced over. It would be a light wrapper over `Variable._coarsen_reshape` It make https://github.com/pydata/xarray/discussions/5119, and #2419 really easy without having to go through a MultiIndex+unstack operation which is somewhat arcane. ``` python ds.coarsen(time=12).construct(dim=""year"").rename({""time"": ""month""}) ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5454/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 664595680,MDU6SXNzdWU2NjQ1OTU2ODA=,4260,use matplotlib's categorical axis features,2448579,closed,0,,,0,2020-07-23T16:01:13Z,2021-06-21T17:45:39Z,2021-06-21T17:45:39Z,MEMBER,,,," **Is your feature request related to a problem? Please describe.** xarray currently doesn't allow plotting against coordinates with string labels for example. **Describe the solution you'd like** Use matplotlib's categorical axis support. Example: https://matplotlib.org/gallery/lines_bars_and_markers/categorical_variables.html This may be the only place a change is required: https://github.com/pydata/xarray/blob/4e893317240ed1a80e65ea2de107e9179bb65446/xarray/plot/utils.py#L572-L608 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4260/reactions"", ""total_count"": 8, ""+1"": 8, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 502149236,MDU6SXNzdWU1MDIxNDkyMzY=,3371,Add xr.unify_chunks top level method,2448579,closed,0,,,4,2019-10-03T15:49:09Z,2021-06-16T14:56:59Z,2021-06-16T14:56:58Z,MEMBER,,,,"This should handle multiple DataArrays and Datasets. Implemented in #3276 as `Dataset.unify_chunks` and `DataArray.unify_chunks`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3371/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 891200849,MDU6SXNzdWU4OTEyMDA4NDk=,5298,0.18.1 patch release?,2448579,closed,0,,,12,2021-05-13T16:50:30Z,2021-05-19T20:47:13Z,2021-05-19T06:53:13Z,MEMBER,,,,"I think we should issue a patch release soon. - [x] `xarray.map_blocks` will break with new dask,distributed releases this week since we don't have a released version with https://github.com/pydata/xarray/pull/5271 - [x] #5296: fix for regression reported in #5295 - [x] #5300: fix the error message in #5291 so that it is more helpful. It's a key entry point for new users. - [x] #5299 RTD build is broken - [ ] #5329 should throw `FileNotFoundError` error on nonexistent file? Nice to have - [ ] #5252 zarr region - [ ] #5045 assigning to subset - [x] #5239 drop_duplicates - [x] #5288 invalid reference date error Thoughts? Volunteers?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5298/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 891240764,MDU6SXNzdWU4OTEyNDA3NjQ=,5299,failing RTD build,2448579,closed,0,,,1,2021-05-13T17:50:37Z,2021-05-14T01:04:22Z,2021-05-14T01:04:22Z,MEMBER,,,,"The RTD build is failing on all PRs with ``` Sphinx parallel build error: nbsphinx.NotebookError: UndefinedError in examples/ERA5-GRIB-example.ipynb: 'nbformat.notebooknode.NotebookNode object' has no attribute 'tags' ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5299/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 870292042,MDU6SXNzdWU4NzAyOTIwNDI=,5232,release v0.18.0,2448579,closed,0,,,60,2021-04-28T19:43:30Z,2021-05-06T21:15:02Z,2021-05-06T19:25:51Z,MEMBER,,,,"As discussed in the meeting, we should issue a release soon with the new backend refactor and the new docs theme. Here's a list of blockers: - [x] #5231 - [x] #5073 - [x] #5235 Would be nice and look done: - [x] #5244 - [x] #5258 - [x] #5101 - [x] ~#4866~ (we should let this sit on master for a while to find bugs) - [x] #4902 - [x] ~#4972~ (this should probably also sit on master for a while) - [x] #5227 - [x] #4740 - [x] #5149 Somewhat important, but no PR yet: - [x] ~#5175~ (as pointed out by @shoyer, this is really a new feature, not a regression, it can wait) @TomNicholas and @alexamici volunteered to handle this. I can be online at release time to help with things if needed. Release instructions are here: https://github.com/pydata/xarray/blob/master/HOW_TO_RELEASE.md IIRC they'll need to be added to the PyPI list and RTD list. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5232/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 501108453,MDU6SXNzdWU1MDExMDg0NTM=,3363,user-friendly additions for dask usage,2448579,closed,0,,,3,2019-10-01T19:48:27Z,2021-04-19T03:34:18Z,2021-04-19T03:34:18Z,MEMBER,,,,"Any thoughts on adding 1. `.chunksize` or `.nbytes_chunk` 2. `.ntasks` : this would be `len(obj.__dask_graph__())`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3363/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 654889988,MDU6SXNzdWU2NTQ4ODk5ODg=,4215,setting variables named in CF attributes as coordinate variables,2448579,closed,0,,,5,2020-07-10T16:17:08Z,2021-04-19T03:32:02Z,2021-04-19T03:32:02Z,MEMBER,,,,"This came up in #2844 by @DWesl (see also #3689) Currently we have `decode_coords` which sets variables named in `attrs[""coordinates""]` as coordinate variables. There are a number of other CF attributes that can contain variable names. 1. bounds 1. grid_mapping 1. ancillary_variables 1. cell_measures 1. maybe more? As in #3689 it's hard to see why a lot of these variables named in these attributes would be useful as ""data variables"". **Question**: Should we allow `decode_coords` to control whether variables mentioned in these attributes are set as coordinate variables? cc @jthielen ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4215/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 850473442,MDU6SXNzdWU4NTA0NzM0NDI=,5113,docs sidebar formatting has changed,2448579,closed,0,,,1,2021-04-05T16:06:43Z,2021-04-19T02:35:34Z,2021-04-19T02:35:34Z,MEMBER,,,," **What happened**: The formatting of section headings ""for users"", ""community"" etc. has changed: https://xarray.pydata.org/en/latest/ ![image](https://user-images.githubusercontent.com/2448579/113595647-86250a80-95f6-11eb-8dc3-804e338c8439.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 847014702,MDU6SXNzdWU4NDcwMTQ3MDI=,5098,open_dataset regression,2448579,closed,0,35919497,,2,2021-03-31T17:32:03Z,2021-04-15T12:11:34Z,2021-04-15T12:11:34Z,MEMBER,,,," **What happened**: `open_dataset` doesn't replace `~` any more **What you expected to happen**: should replace `~` for backward compatibility **Minimal Complete Verifiable Example**: ```python import xarray as xr da = xr.DataArray([1, 2, 3]) da.to_netcdf(""~/bug_report.nc"") xr.open_dataarray(""~/bug_report.nc"") ``` **Anything else we need to know?**: works on 0.17.0, fails on master ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5098/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 636666706,MDU6SXNzdWU2MzY2NjY3MDY=,4146,sparse upstream-dev test failures,2448579,closed,0,,,4,2020-06-11T02:20:11Z,2021-03-17T23:10:45Z,2020-06-16T16:00:10Z,MEMBER,,,,"Full log here: https://dev.azure.com/xarray/xarray/_build/results?buildId=3023&view=logs&jobId=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&j=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&t=175181ee-1928-5a6b-f537-168f7a8b7c2d Here are three of the errors: ``` /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/sparse/_coo/umath.py:739: SystemError _ test_variable_method[obj.where(*(), **{'cond': \n})-True] _ TypeError: expected dtype object, got 'numpy.dtype[uint64]' ``` ``` def _match_coo(*args, **kwargs): """""" Matches the coordinates for any number of input :obj:`COO` arrays. Equivalent to ""sparse"" broadcasting for all arrays. Parameters ---------- args : Tuple[COO] The input :obj:`COO` arrays. return_midx : bool Whether to return matched indices or matched arrays. Matching only supported for two arrays. ``False`` by default. cache : dict Cache of things already matched. No cache by default. Returns ------- matched_idx : List[ndarray] The indices of matched elements in the original arrays. Only returned if ``return_midx`` is ``True``. matched_arrays : List[COO] The expanded, matched :obj:`COO` objects. Only returned if ``return_midx`` is ``False``. """""" from .core import COO from .common import linear_loc cache = kwargs.pop(""cache"", None) return_midx = kwargs.pop(""return_midx"", False) broadcast_shape = kwargs.pop(""broadcast_shape"", None) if kwargs: linear = [idx[s] for idx, s in zip(linear, sorted_idx)] > matched_idx = _match_arrays(*linear) E SystemError: CPUDispatcher() returned a result with an error set ``` ``` _______________________________ test_dask_token ________________________________ @requires_dask def test_dask_token(): import dask s = sparse.COO.from_numpy(np.array([0, 0, 1, 2])) # https://github.com/pydata/sparse/issues/300 s.__dask_tokenize__ = lambda: dask.base.normalize_token(s.__dict__) a = DataArray(s) t1 = dask.base.tokenize(a) t2 = dask.base.tokenize(a) t3 = dask.base.tokenize(a + 1) assert t1 == t2 assert t3 != t2 assert isinstance(a.data, sparse.COO) ac = a.chunk(2) t4 = dask.base.tokenize(ac) t5 = dask.base.tokenize(ac + 1) assert t4 != t5 > assert isinstance(ac.data._meta, sparse.COO) E AssertionError: assert False E + where False = isinstance(array([], dtype=int64), ) E + where array([], dtype=int64) = dask.array, shape=(4,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>._meta E + where dask.array, shape=(4,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray> = \ndask.array, shape=(4,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>\nDimensions without coordinates: dim_0.data E + and = sparse.COO ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4146/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 819241806,MDU6SXNzdWU4MTkyNDE4MDY=,4980,fix bottleneck + Dask 1D rolling operations,2448579,closed,0,,,1,2021-03-01T20:38:34Z,2021-03-01T20:39:28Z,2021-03-01T20:39:27Z,MEMBER,,,,"Just as a reminder. Right now all rolling operations with dask arrays use `.construct().reduce()`. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4980/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 806436724,MDU6SXNzdWU4MDY0MzY3MjQ=,4894,release 0.17.0,2448579,closed,0,,,14,2021-02-11T14:24:18Z,2021-02-26T20:16:16Z,2021-02-26T20:16:16Z,MEMBER,,,,"Quoting @spencerkclark email: > Somewhat related to this issue, would others on the core team be open to making a new xarray release soon? In addition to the license change, this cftime update unintentionally introduced a change* that broke our encoding logic for cftime dates in a subtle way. I recently fixed this in pydata/xarray#4871, but until a new xarray release is made, this issue makes it dangerous to use the latest non-GPL-licensed version of cftime. These three seem like high-priority fixes. Thoughts? - [x] #4884 - [x] #4893 - [ ] #4879 ? - [x] #4911 - [x] #4179 / #4907 Nice to have - [ ] #4740","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4894/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 685590739,MDU6SXNzdWU2ODU1OTA3Mzk=,4373,Add Dataset.plot.quiver,2448579,closed,0,,,0,2020-08-25T15:39:37Z,2021-02-19T14:21:45Z,2021-02-19T14:21:45Z,MEMBER,,,,"I think it would be nice to add a quiver plot function. I got this far in my current project: ``` python @xarray.plot.dataset_plot._dsplot def quiver(ds, x, y, ax, u, v, **kwargs): from xarray import broadcast if x is None or y is None or u is None or v is None: raise ValueError(""Must specify x, y, u, v for quiver plots."") # matplotlib autoscaling algorithm scale = kwargs.pop(""scale"", None) if scale is None: npts = ds.dims[x] * ds.dims[y] # crude auto-scaling # scale is typical arrow length as a multiple of the arrow width scale = ( 1.8 * ds.to_array().median().values * np.maximum(10, np.sqrt(npts)) ) # / span ds = ds.squeeze() x, y, u, v = broadcast(ds[x], ds[y], ds[u], ds[v]) # no support for these kwargs right now. kwargs.pop(""cmap_params"") kwargs.pop(""hue"") kwargs.pop(""hue_style"") hdl = ax.quiver(x.values, y.values, u.values, v.values, scale=scale, **kwargs) return hdl ``` The autoscaling logic is quite crude; I tried to copy what matplotlib does but got somewhat confused. To get faceting to work properly, we'll need to estimate `scale` around here: https://github.com/pydata/xarray/blob/a36d0a1d4657c848dcdd76d0ecb9c783ad464057/xarray/plot/facetgrid.py#L339-L343","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4373/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 674414304,MDU6SXNzdWU2NzQ0MTQzMDQ=,4320,html repr doesn't work in some sphinx themes,2448579,closed,0,,,1,2020-08-06T15:45:54Z,2021-01-31T03:34:55Z,2021-01-31T03:34:54Z,MEMBER,,,,"Downstream issue: https://github.com/xarray-contrib/cf-xarray/issues/57 Example: no reprs displayed in https://cf-xarray.readthedocs.io/en/latest/examples/introduction.html @benbovy's diagnosis: > It looks like bootstrap 4 (used by sphinx-book-theme) forces all html elements with hidden attributes to be actually hidden (source), so the hack in pydata/xarray#4053 does not work here (the result is even worse). > I guess that a workaround would be to add some custom CSS such as .xr-wrap { display: block !important }, assuming that custom CSS is loaded after Bootstrap's CSS. Not ideal, though, it looks like a hack on top of another hack.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4320/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 771295760,MDU6SXNzdWU3NzEyOTU3NjA=,4713,mamba on RTD,2448579,closed,0,,,6,2020-12-19T03:43:50Z,2020-12-21T21:19:47Z,2020-12-21T21:19:47Z,MEMBER,,,,"RTD now supports using mamba instead of conda using a feature flag : https://docs.readthedocs.io/en/latest/guides/feature-flags.html#available-flags Maybe it's worth seeing if this will reduce docs build time on CI? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4713/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 766926186,MDU6SXNzdWU3NjY5MjYxODY=,4693,codecov coverage dropped,2448579,closed,0,,,2,2020-12-14T21:03:47Z,2020-12-20T00:48:43Z,2020-12-20T00:48:43Z,MEMBER,,,,"Looks like something went wrong when #4408 was merged (https://github.com/pydata/xarray/commit/23dc2fc9f2785c348ff821bf2da61dfa2206d283) ![image](https://user-images.githubusercontent.com/2448579/102135305-04298f80-3e15-11eb-9f6f-e44238a414ac.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4693/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 484955475,MDU6SXNzdWU0ODQ5NTU0NzU=,3263,test_sparse doesn't work with pytest-xdist,2448579,closed,0,,,0,2019-08-25T17:30:51Z,2020-12-17T22:33:46Z,2020-12-17T22:33:46Z,MEMBER,,,,"`pytest -n 4 test_sparse.py` results in ``` ====================================================================================== ERRORS ====================================================================================== _______________________________________________________________________________ ERROR collecting gw0 _______________________________________________________________________________ Different tests were collected between gw1 and gw0. The difference is: --- gw1 +++ gw0 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] _______________________________________________________________________________ ERROR collecting gw2 _______________________________________________________________________________ Different tests were collected between gw1 and gw2. The difference is: --- gw1 +++ gw2 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -118,7 +118,7 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2]})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.shift(*(), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sortby(*('x',), **{'ascending': False})-True] -xarray/tests/test_sparse.py::test_dataarray_method[obj.stack(*(), **{'z': {'y', 'x'}})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.stack(*(), **{'z': {'x', 'y'}})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.transpose(*(), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.broadcast_equals(*(\n,), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.equals(*(\n,), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] _______________________________________________________________________________ ERROR collecting gw3 _______________________________________________________________________________ Different tests were collected between gw1 and gw3. The difference is: --- gw1 +++ gw3 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] ================================================================================= warnings summary ================================================================================= /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import defaultdict, Iterator /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import defaultdict, deque, OrderedDict, Mapping, Set -- Docs: https://docs.pytest.org/en/latest/warnings.html ======================================================================= ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 665250379,MDU6SXNzdWU2NjUyNTAzNzk=,4266,docstrings for groupby reductions don't work in IPython,2448579,closed,0,,,2,2020-07-24T15:36:26Z,2020-11-25T23:55:18Z,2020-11-25T23:55:18Z,MEMBER,,,,"In IPython, ```python ds = xr.tutorial.load_dataset(""air_temperature"") ds.air.groupby(""lat"").mean? ``` gives output ``` Object `mean` not found. ``` BUT this does work. ``` python print(ds.air.groupby(""lat"").mean.__doc__) ``` ``` Reduce this DataArrayGroupBy's data by applying `mean` along some dimension(s). Parameters ---------- dim : str or sequence of str, optional Dimension(s) over which to apply `mean`. ... ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4266/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 722539100,MDU6SXNzdWU3MjI1MzkxMDA=,4515,show dimension coordinates at top of coordinates repr,2448579,closed,0,,,1,2020-10-15T17:44:28Z,2020-11-06T18:49:55Z,2020-11-06T18:49:55Z,MEMBER,,,," **Is your feature request related to a problem? Please describe.** I have datasets with lots of non-dim coord variables. Its annoying to search through and look at the dimension coordinates to get an idea of what subset of data I am looking at. ![image](https://user-images.githubusercontent.com/2448579/96166307-076ad280-0edb-11eb-8e87-470771b0a289.png) **Describe the solution you'd like** I think we should show dimension coordinate variables at the top of the coordinates repr. Example code ``` python ds = xr.Dataset() ds.coords[""as""] = 10 ds[""var""] = xr.DataArray(np.ones((10,)), dims=""x"", coords={""x"": np.arange(10)}) ds ``` ![image](https://user-images.githubusercontent.com/2448579/96166698-8c55ec00-0edb-11eb-825b-dabd460fe58b.png) Related #4409 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4515/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 435330735,MDU6SXNzdWU0MzUzMzA3MzU=,2909,docs questions from new user,2448579,closed,0,,,2,2019-04-19T23:29:35Z,2020-10-13T04:06:14Z,2020-10-13T04:06:14Z,MEMBER,,,,"A relatively new user here gave me an incomplete list of things they found to be confusing in the docs: - ~What is xarray's data model?~ opened #2913 - [x] What is split-apply-combine? - [x] What are dummy dimensions and `np.newaxis` - [x] quick-overiew: Index so that you extract a subset! - [ ] too many references to pandas! - [x] do I need to know pandas to use xarray? - [ ] what is the difference between attrs and encoding? How do I use them properly/correctly? - [ ] why `isnull` and `dropna`: why `dropna` instead of `dropnull` - [x] `compute` docstring doesn't mention computation! - [x] compute vs persist vs load: what's the difference - [ ] do I get parallelization for free? - [ ] do I start a dask cluster first? - [x] dask page links to shoyers blog but that doesn't explicitly start a cluster. I'm crossing out things that I've started to fix. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2909/reactions"", ""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 589583977,MDU6SXNzdWU1ODk1ODM5Nzc=,3908,"check index invariants in test_dataset, test_dataarray",2448579,closed,0,,,5,2020-03-28T14:07:17Z,2020-10-13T04:03:58Z,2020-10-13T04:03:58Z,MEMBER,,,,"There are a large number of tests in `test_dataarray.py` and `test_dataset.py` that use the `assert actual.equals(expected)` pattern. We should switch these to be `assert_equal(actual, expected)` so that we run the invariant checks: https://github.com/pydata/xarray/blob/acf7d4157ca44f05c85a92d1b914b68738988773/xarray/tests/__init__.py#L150-L154 Seems like an easy use of regexes for someone that knows them (i.e. not me ;))","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3908/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 550355524,MDU6SXNzdWU1NTAzNTU1MjQ=,3698,dask.optimize on xarray objects,2448579,closed,0,,,5,2020-01-15T18:29:18Z,2020-09-20T05:21:57Z,2020-09-20T05:21:57Z,MEMBER,,,,"I am trying to call `dask.optimize` on a xarray object before the graph gets too big. But get weird errors. Simple examples below. All examples work if I remove the `dask.optimize` step. cc @mrocklin @shoyer ### This works with dask arrays: ``` python a = dask.array.ones((10,5), chunks=(1,3)) a = dask.optimize(a)[0] a.compute() ``` ### It works when a dataArray is constructed using a dask array ``` python da = xr.DataArray(a) da = dask.optimize(da)[0] da.compute() ``` ### but fails when creating a DataArray with a numpy array and then chunking it :man_shrugging: ``` python da = xr.DataArray(a.compute()).chunk({""dim_0"": 5}) da = dask.optimize(da)[0] da.compute() ``` fails with error ``` python --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in 1 da = xr.DataArray(a.compute()).chunk({""dim_0"": 5}) 2 da = dask.optimize(da)[0] ----> 3 da.compute() ~/python/xarray/xarray/core/dataarray.py in compute(self, **kwargs) 838 """""" 839 new = self.copy(deep=False) --> 840 return new.load(**kwargs) 841 842 def persist(self, **kwargs) -> ""DataArray"": ~/python/xarray/xarray/core/dataarray.py in load(self, **kwargs) 812 dask.array.compute 813 """""" --> 814 ds = self._to_temp_dataset().load(**kwargs) 815 new = self._from_temp_dataset(ds) 816 self._variable = new._variable ~/python/xarray/xarray/core/dataset.py in load(self, **kwargs) 659 660 # evaluate all the dask arrays simultaneously --> 661 evaluated_data = da.compute(*lazy_data.values(), **kwargs) 662 663 for k, data in zip(lazy_data, evaluated_data): ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/base.py in compute(*args, **kwargs) 434 keys = [x.__dask_keys__() for x in collections] 435 postcomputes = [x.__dask_postcompute__() for x in collections] --> 436 results = schedule(dsk, keys, **kwargs) 437 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)]) 438 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, pool, **kwargs) 79 get_id=_thread_get_id, 80 pack_exception=pack_exception, ---> 81 **kwargs 82 ) 83 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs) 484 _execute_task(task, data) # Re-execute locally 485 else: --> 486 raise_exception(exc, tb) 487 res, worker_id = loads(res_info) 488 state[""cache""][key] = res ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in reraise(exc, tb) 314 if exc.__traceback__ is not tb: 315 raise exc.with_traceback(tb) --> 316 raise exc 317 318 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in execute_task(key, task_info, dumps, loads, get_id, pack_exception) 220 try: 221 task, data = loads(task_info) --> 222 result = _execute_task(task, data) 223 id = get_id() 224 result = dumps((result, id)) ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/core.py in _execute_task(arg, cache, dsk) 117 func, args = arg[0], arg[1:] 118 args2 = [_execute_task(a, cache) for a in args] --> 119 return func(*args2) 120 elif not ishashable(arg): 121 return arg TypeError: string indices must be integers ``` ### And a different error when rechunking a dask-backed DataArray ``` python da = xr.DataArray(a).chunk({""dim_0"": 5}) da = dask.optimize(da)[0] da.compute() ``` ``` python --------------------------------------------------------------------------- IndexError Traceback (most recent call last) in 1 da = xr.DataArray(a).chunk({""dim_0"": 5}) 2 da = dask.optimize(da)[0] ----> 3 da.compute() ~/python/xarray/xarray/core/dataarray.py in compute(self, **kwargs) 838 """""" 839 new = self.copy(deep=False) --> 840 return new.load(**kwargs) 841 842 def persist(self, **kwargs) -> ""DataArray"": ~/python/xarray/xarray/core/dataarray.py in load(self, **kwargs) 812 dask.array.compute 813 """""" --> 814 ds = self._to_temp_dataset().load(**kwargs) 815 new = self._from_temp_dataset(ds) 816 self._variable = new._variable ~/python/xarray/xarray/core/dataset.py in load(self, **kwargs) 659 660 # evaluate all the dask arrays simultaneously --> 661 evaluated_data = da.compute(*lazy_data.values(), **kwargs) 662 663 for k, data in zip(lazy_data, evaluated_data): ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/base.py in compute(*args, **kwargs) 434 keys = [x.__dask_keys__() for x in collections] 435 postcomputes = [x.__dask_postcompute__() for x in collections] --> 436 results = schedule(dsk, keys, **kwargs) 437 return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)]) 438 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, pool, **kwargs) 79 get_id=_thread_get_id, 80 pack_exception=pack_exception, ---> 81 **kwargs 82 ) 83 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in get_async(apply_async, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, **kwargs) 484 _execute_task(task, data) # Re-execute locally 485 else: --> 486 raise_exception(exc, tb) 487 res, worker_id = loads(res_info) 488 state[""cache""][key] = res ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in reraise(exc, tb) 314 if exc.__traceback__ is not tb: 315 raise exc.with_traceback(tb) --> 316 raise exc 317 318 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/local.py in execute_task(key, task_info, dumps, loads, get_id, pack_exception) 220 try: 221 task, data = loads(task_info) --> 222 result = _execute_task(task, data) 223 id = get_id() 224 result = dumps((result, id)) ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/core.py in _execute_task(arg, cache, dsk) 117 func, args = arg[0], arg[1:] 118 args2 = [_execute_task(a, cache) for a in args] --> 119 return func(*args2) 120 elif not ishashable(arg): 121 return arg ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/array/core.py in concatenate3(arrays) 4305 if not ndim: 4306 return arrays -> 4307 chunks = chunks_from_arrays(arrays) 4308 shape = tuple(map(sum, chunks)) 4309 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/array/core.py in chunks_from_arrays(arrays) 4085 4086 while isinstance(arrays, (list, tuple)): -> 4087 result.append(tuple([shape(deepfirst(a))[dim] for a in arrays])) 4088 arrays = arrays[0] 4089 dim += 1 ~/miniconda3/envs/dcpy_updated/lib/python3.7/site-packages/dask/array/core.py in (.0) 4085 4086 while isinstance(arrays, (list, tuple)): -> 4087 result.append(tuple([shape(deepfirst(a))[dim] for a in arrays])) 4088 arrays = arrays[0] 4089 dim += 1 IndexError: tuple index out of range ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3698/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 630062936,MDU6SXNzdWU2MzAwNjI5MzY=,4120,coarsen deletes attrs on original object,2448579,closed,0,,,13,2020-06-03T14:50:16Z,2020-09-09T19:30:42Z,2020-09-09T18:56:36Z,MEMBER,,,," #### MCVE Code Sample ```python # Your code here import xarray as xr ds = xr.tutorial.load_dataset(""air_temperature"") ds2 = xr.tutorial.load_dataset(""air_temperature"") xr.testing.assert_identical(ds, ds2) # passes ds.coarsen(lat=5).mean() xr.testing.assert_identical(ds, ds2) # fails ``` Bug: `coarsen` reductions are modifying the original dataset by deleting all attributes ``` AssertionErrorTraceback (most recent call last) in 48 xr.testing.assert_identical(ds, ds2) 49 ds.coarsen(lat=5).mean() ---> 50 xr.testing.assert_identical(ds, ds2) ~/work/python/xarray/xarray/testing.py in assert_identical(a, b) 87 assert a.identical(b), formatting.diff_array_repr(a, b, ""identical"") 88 elif isinstance(a, (Dataset, Variable)): ---> 89 assert a.identical(b), formatting.diff_dataset_repr(a, b, ""identical"") 90 else: 91 raise TypeError(""{} not supported by assertion comparison"".format(type(a))) AssertionError: Left and right Dataset objects are not identical Differing coordinates: L * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0 R * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0 standard_name: latitude long_name: Latitude units: degrees_north axis: Y Differing data variables: L air (time, lat, lon) float32 241.2 242.5 243.5 ... 296.49 296.19 295.69 R air (time, lat, lon) float32 241.2 242.5 243.5 ... 296.49 296.19 295.69 long_name: 4xDaily Air temperature at sigma level 995 units: degK precision: 2 GRIB_id: 11 GRIB_name: TMP var_desc: Air temperature dataset: NMC Reanalysis level_desc: Surface statistic: Individual Obs parent_stat: Other actual_range: [185.16 322.1 ] ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4120/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 684930038,MDU6SXNzdWU2ODQ5MzAwMzg=,4372,Set `allow_rechunk=True` in `apply_ufunc`,2448579,closed,0,,,10,2020-08-24T20:02:50Z,2020-09-09T19:00:17Z,2020-09-09T19:00:17Z,MEMBER,,,," **What happened**: `blockwise` calls `unify_chunks` by default but `apply_gufunc` does not; so we have a regression in `apply_ufunc` now that we've switched from `blockwise` to `apply_gufunc`. **Minimal Complete Verifiable Example**: ```python import operator a = xr.DataArray(np.arange(10), dims=(""a"")).chunk({""a"": 2}) b = xr.DataArray(np.arange(10), dims=(""a"")).chunk({""a"": 4}) xr.apply_ufunc(operator.add, a, b, dask=""parallelized"", output_dtypes=[a.dtype]).compute() ``` raises ``` ValueError: Dimension `'__loopdim0__'` with different chunksize present ``` on master but works with 0.16.0 I think we need to do `dask_gufunc_kwargs.setdefault(""allow_rechunk"", True)` If we want to avoid that, we'll need to go through a deprecation cycle.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4372/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 665232266,MDU6SXNzdWU2NjUyMzIyNjY=,4265,cftime plotting fails on upstream-dev,2448579,closed,0,,,5,2020-07-24T15:07:44Z,2020-07-27T13:13:48Z,2020-07-26T19:04:55Z,MEMBER,,,,"seen in https://dev.azure.com/xarray/xarray/_build/results?buildId=3365&view=logs&jobId=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&j=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&t=175181ee-1928-5a6b-f537-168f7a8b7c2d ``` =========================== short test summary info ============================ FAILED xarray/tests/test_plot.py::TestCFDatetimePlot::test_cfdatetime_line_plot FAILED xarray/tests/test_plot.py::TestCFDatetimePlot::test_cfdatetime_pcolormesh_plot FAILED xarray/tests/test_plot.py::TestCFDatetimePlot::test_cfdatetime_contour_plot ``` e.g. ``` =================================== FAILURES =================================== _________________ TestCFDatetimePlot.test_cfdatetime_line_plot _________________ self = def test_cfdatetime_line_plot(self): E ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 1. /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/matplotlib/transforms.py:943: ValueError ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4265/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 663833847,MDU6SXNzdWU2NjM4MzM4NDc=,4249,RTD PR builds are timing out,2448579,closed,0,,,1,2020-07-22T15:04:22Z,2020-07-22T21:17:59Z,2020-07-22T21:17:59Z,MEMBER,,,,"See https://readthedocs.org/projects/xray/builds/ There's no useful information in the logs AFAICT: e.g. https://readthedocs.org/projects/xray/builds/11504571/","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 612785915,MDU6SXNzdWU2MTI3ODU5MTU=,4031,0.16.0 release,2448579,closed,0,,,9,2020-05-05T17:53:26Z,2020-07-14T17:54:31Z,2020-07-14T17:54:31Z,MEMBER,,,,"It'd be nice to issue a release soon. We should decide if this is a minor 0.15.2 or major 0.16.0 Please edit this list as you wish. **Must-have** - [x] #3936 multiple dims argmin, argmax - [x] #4019, #4088; MultiIndex to sparse DataArray bug - [x] #3922 fix dask array handling in idxmax, idxmin - [x] #4009, #4173 combine_attrs with open_mfdataset - [x] #3824 transpose coords (if major release) - [x] #3926 remove old autocombine - [x] #4135 nD dask arrays in idxmax/min **Nice to have** - [x] #4049, #4094 unstacking merge bug - [ ] #4022 apply_ufunc meta vectorize - [ ] #3924 interpolating to coordinates with nans - [ ] #3925 sel along unindexed 1d coordinates - [ ] #3993 replace dim with coord in integrate - [ ] #3594 unit support with pint - [ ] #4003 mfdataset with zarr - [x] #3905 length of dataarray reprs - [x] #3847 error summary for assert_allclose - [x] #4033 infer_freq - [x] #3938 plotting with multiindex - [x] #3816 map_blocks template - [x] #3976 in-place replacement error message - [x] #3975 pint dataset - [x] #4018 netdf3 dtype coercion - [ ] #3985 groupby multi-index bug (ref discussion below & in the issue)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4031/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 636665269,MDU6SXNzdWU2MzY2NjUyNjk=,4145,Fix matplotlib in upstream-dev test config,2448579,closed,0,,,4,2020-06-11T02:15:52Z,2020-06-12T09:11:31Z,2020-06-12T09:11:31Z,MEMBER,,,,"From @keewis comment in #4138 > I just noticed that the rackcdn.org repository doesn't have matplotlib>=3.2.0, so since about late February we don't test against matplotlib upstream anymore.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4145/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 624424922,MDU6SXNzdWU2MjQ0MjQ5MjI=,4093,rename_dims to an existing dimension name,2448579,closed,0,,,2,2020-05-25T18:00:28Z,2020-05-25T19:35:09Z,2020-05-25T19:35:08Z,MEMBER,,,,"``` ds = xr.Dataset({""a"": (""x"", np.arange(10)), ""b"": (""y"", np.arange(10))}) ds.rename_dims({""x"": ""y""}) ``` ``` ValueError: Cannot rename x to y because y already exists. Try using swap_dims instead. ``` This should be possible since both `x` and `y` have the same lengths. Thoughts? For indexed dimensions, should we check that the indexes are equal? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4093/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 613579881,MDU6SXNzdWU2MTM1Nzk4ODE=,4041,expanded HTML repr when opening notebook,2448579,closed,0,,,16,2020-05-06T20:06:38Z,2020-05-20T17:06:40Z,2020-05-20T17:06:40Z,MEMBER,,,,"When I open a notebook, the new HTML repr is ""expanded"": ![image](https://user-images.githubusercontent.com/2448579/81222973-bc1bd200-8fd4-11ea-8afb-84d706ae9b56.png) I'm running ``` jupyter core : 4.6.3 jupyter-notebook : 6.0.3 qtconsole : 4.7.2 ipython : 7.13.0 ipykernel : 5.2.0 jupyter client : 6.1.2 jupyter lab : 2.1.0 nbconvert : 5.6.1 ipywidgets : 7.5.1 nbformat : 5.0.4 traitlets : 4.3.3 ``` and see this behaviour with both the notebook and jupyterlab interface; on both chrome and firefox (both on linux). Is anyone else seeing this? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4041/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 476222321,MDU6SXNzdWU0NzYyMjIzMjE=,3178,type annotations make docs confusing,2448579,closed,0,,,17,2019-08-02T14:57:33Z,2020-05-19T16:49:26Z,2020-05-19T16:49:26Z,MEMBER,,,,"The annotations make this signature basically unreadable to anyone not familiar with them. ![image](https://user-images.githubusercontent.com/2448579/62378906-57a78000-b535-11e9-97ab-4c068fc58d1e.png) Is there a way to hide them in the documentation? Or at least change formatting so it's clearer what the function arguments are? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3178/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 617579699,MDU6SXNzdWU2MTc1Nzk2OTk=,4056,flake8 failure,2448579,closed,0,,,1,2020-05-13T16:16:20Z,2020-05-13T17:35:46Z,2020-05-13T17:35:46Z,MEMBER,,,,"flake8 is failing on master (https://dev.azure.com/xarray/xarray/_build/results?buildId=2820&view=logs&jobId=a577607c-d99b-546f-eeb4-2341e9a21630&j=a577607c-d99b-546f-eeb4-2341e9a21630&t=7308a173-bf34-5af1-b6d9-30c4d79bebeb) with ``` ========================== Starting Command Output =========================== /bin/bash --noprofile --norc /home/vsts/work/_temp/e6322963-dd1c-4887-ba6a-2aa7ec888f4c.sh ./xarray/backends/memory.py:43:32: E741 ambiguous variable name 'l' ./xarray/backends/common.py:244:32: E741 ambiguous variable name 'l' ##[error]Bash exited with code '1'. Finishing: flake8 lint checks ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4056/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 584429748,MDU6SXNzdWU1ODQ0Mjk3NDg=,3867,macos py38 CI failing,2448579,closed,0,,,3,2020-03-19T13:54:10Z,2020-03-29T22:13:26Z,2020-03-29T22:13:26Z,MEMBER,,,,"`import matplotlib` is failing when it imports `PIL` ```python E ImportError: dlopen(/usr/local/miniconda/envs/xarray-tests/lib/python3.8/site-packages/PIL/_imaging.cpython-38-darwin.so, 2): Library not loaded: @rpath/libwebp.7.dylib E Referenced from: /usr/local/miniconda/envs/xarray-tests/lib/libtiff.5.dylib E Reason: Incompatible library version: libtiff.5.dylib requires version 9.0.0 or later, but libwebp.7.dylib provides version 8.0.0 /usr/local/miniconda/envs/xarray-tests/lib/python3.8/site-packages/PIL/Image.py:69: ImportError ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3867/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 570190199,MDU6SXNzdWU1NzAxOTAxOTk=,3796,RTD failing yet again,2448579,closed,0,,,7,2020-02-24T22:35:52Z,2020-03-24T22:23:00Z,2020-03-24T22:23:00Z,MEMBER,,,,"memory consumption errors as usual. @keewis I remember you had an idea for using pip instead of conda?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3796/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 584472210,MDU6SXNzdWU1ODQ0NzIyMTA=,3869,0.15.1 release,2448579,closed,0,,,12,2020-03-19T14:55:31Z,2020-03-23T23:07:12Z,2020-03-23T21:11:15Z,MEMBER,,,,"It'd be nice to issue a release by the end of March (two months since previous release) Please edit this list as you wish. **Must-have** - [x] #3862 error when assigning to IndexVariable.values - [x] #3840 delete index when deleting coordinate variable - [x] #3874 un-xfail the tests from #3808 (https://github.com/pydata/xarray/issues/3751#issuecomment-599231986) - [x] #3833, #3870 HTML repr with non-str keys fix (since it is now on by default) **Nice to have** - [ ] #3816 map_blocks template - [x] #3817 map_blocks new unindexed dimension - [ ] #3818 map_blocks dask args - [ ] #3847 assert_allclose diff - [x] #3826 ellipsis in stack - [x] #3836 where docstring - [ ] some progress on #3868 - [ ] #3594 unit support - [ ] #3871 idxmin, idxmax ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3869/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 559873728,MDU6SXNzdWU1NTk4NzM3Mjg=,3751,more upstream-dev cftime failures,2448579,closed,0,,,20,2020-02-04T17:36:03Z,2020-03-22T15:01:36Z,2020-03-13T06:14:41Z,MEMBER,,,,"https://dev.azure.com/xarray/xarray/_build/results?buildId=2116&view=logs&jobId=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&j=2280efed-fda1-53bd-9213-1fa8ec9b4fa8&t=175181ee-1928-5a6b-f537-168f7a8b7c2d 46 failed tests but they all seem to result from the same TypeError ``` =================================== FAILURES =================================== ______________ test_sel_date_scalar_nearest[365_day-sel_kwargs0] _______________ da = 0002-02-01 00:00:00], dtype='object') sel_kwargs = {'method': 'nearest'} @requires_cftime @pytest.mark.parametrize( ""sel_kwargs"", [{""method"": ""nearest""}, {""method"": ""nearest"", ""tolerance"": timedelta(days=70)}], ) def test_sel_date_scalar_nearest(da, date_type, index, sel_kwargs): expected = xr.DataArray(2).assign_coords(time=index[1]) > result = da.sel(time=date_type(1, 4, 1), **sel_kwargs) xarray/tests/test_cftimeindex.py:460: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ xarray/core/dataarray.py:1056: in sel ds = self._to_temp_dataset().sel( xarray/core/dataset.py:2056: in sel pos_indexers, new_indexes = remap_label_indexers( xarray/core/coordinates.py:391: in remap_label_indexers pos_indexers, new_indexes = indexing.remap_label_indexers( xarray/core/indexing.py:270: in remap_label_indexers idxr, new_idx = convert_label_indexer(index, label, dim, method, tolerance) xarray/core/indexing.py:189: in convert_label_indexer indexer = index.get_loc( xarray/coding/cftimeindex.py:334: in get_loc return pd.Index.get_loc(self, key, method=method, tolerance=tolerance) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/pandas/core/indexes/base.py:2899: in get_loc indexer = self.get_indexer([key], method=method, tolerance=tolerance) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/pandas/core/indexes/base.py:2992: in get_indexer indexer = self._get_nearest_indexer(target, limit, tolerance) /usr/share/miniconda/envs/xarray-tests/lib/python3.8/site-packages/pandas/core/indexes/base.py:3076: in _get_nearest_indexer left_distances = np.abs(self[left_indexer] - target) xarray/coding/cftimeindex.py:444: in __sub__ return CFTimeIndex(np.array(self) - other) xarray/coding/cftimeindex.py:248: in __new__ assert_all_valid_date_type(data) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ data = TimedeltaIndex(['-59 days'], dtype='timedelta64[ns]', freq=None) def assert_all_valid_date_type(data): import cftime if len(data) > 0: sample = data[0] date_type = type(sample) if not isinstance(sample, cftime.datetime): > raise TypeError( ""CFTimeIndex requires cftime.datetime "" ""objects. Got object of {}."".format(date_type) ) E TypeError: CFTimeIndex requires cftime.datetime objects. Got object of . xarray/coding/cftimeindex.py:206: TypeError ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3751/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 582474355,MDU6SXNzdWU1ODI0NzQzNTU=,3861,CI not running?,2448579,closed,0,,,1,2020-03-16T17:23:13Z,2020-03-17T13:18:07Z,2020-03-17T13:18:07Z,MEMBER,,,,"Looks like the last run was on Thursday: https://dev.azure.com/xarray/xarray/_build?definitionId=1&_a=summary&view=runs No tests have been run for PRs #3826 #3836 #3858 and #3807 despite these having been updated recently. There is a workaround posted at this Azure issue: https://status.dev.azure.com/_event/179641421 but it looks like a fix is coming soon. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3861/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 526754376,MDU6SXNzdWU1MjY3NTQzNzY=,3558,optimizing xarray operations for lazy array equality test,2448579,closed,0,,,0,2019-11-21T18:01:51Z,2020-02-24T18:26:30Z,2020-02-24T18:26:30Z,MEMBER,,,,"TLDR: I think we want `A.sel(x=A.x).equals(A)` to pass lazily. It doesn't do so currently. Currently if I do `A.sel(x=A.x)`, this sticks in a `getitem` call in the dask graph which breaks our lazy array equality optimization. Here's an example ``` python >>> A = xr.DataArray(np.arange(100), dims=""x"", coords={""x"": np.arange(100)}).chunk({""x"": 1}) >>> A.sel(x=A.x).variable.equals(A, equiv=xr.core.duck_array_ops.lazy_array_equiv) None ``` **Questions:** 1. Where is the best place to do this? In `sel` or `isel`? Both? Sticking the following in `sel` makes the above check return `True` which is what we want: ``` python if self._indexes: equals = [] for index in indexers: equals.append(indexers[index].to_index().equals(self._indexes[index])) if all(equals): return self ``` This doesn't handle slice objects though so that makes me think we'd want to add something similar to `isel` too. 2. What is the behaviour we want? `A.sel(x=A.x).equals(A)` or `A.sel(x=A.x) is A`? 3. Doing the latter will mean changing `_to_temp_dataset` and `_from_temp_dataset` which suggests the constraint `A._from_temp_dataset(A._to_temp_dataset()) is A`? But this seems too strong to me. Do we only want to lazily satisfy an `equals` constraint rather than an `identical` constraint? 4. It seems like we'll want to add such short-circuits in many places (I have not checked all of these): `sortby`, `broadcast`, `align`, `reindex` (`transpose` does this now).","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3558/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue