id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 2228319306,I_kwDOAMm_X86E0XRK,8914,swap_dims does not propagate indexes properly,2448579,open,0,,,0,2024-04-05T15:36:26Z,2024-04-05T15:36:27Z,,MEMBER,,,,"### What happened? Found by hypothesis ``` import xarray as xr import numpy as np var = xr.Variable(dims=""2"", data=np.array(['1970-01-01T00:00:00.000000000', '1970-01-01T00:00:00.000000002', '1970-01-01T00:00:00.000000001'], dtype='datetime64[ns]')) var1 = xr.Variable(data=np.array([0], dtype=np.uint32), dims=['1'], attrs={}) state = xr.Dataset() state['2'] = var state = state.stack({""0"": [""2""]}) state['1'] = var1 state['1_'] = var1#.copy(deep=True) state = state.swap_dims({""1"": ""1_""}) xr.testing.assertions._assert_internal_invariants(state, False) ``` This swaps simple pandas indexed dims, but the multi-index that is in the dataset and not affected by the swap_dims op ends up broken. cc @benbovy ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. - [ ] Recent environment — the issue occurs with the latest version of xarray and its dependencies. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8914/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 2136709010,I_kwDOAMm_X85_W5eS,8753,Lazy Loading with `DataArray` vs. `Variable`,2448579,closed,0,,,0,2024-02-15T14:42:24Z,2024-04-04T16:46:54Z,2024-04-04T16:46:54Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/8751
Originally posted by **ilan-gold** February 15, 2024 My goal is to get a dataset from [custom io-zarr backend lazy-loaded](https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#how-to-support-lazy-loading). But when I declare a `DataArray` based on the `Variable` which uses `LazilyIndexedArray`, everything is read in. Is this expected? I specifically don't want to have to use dask if possible. I have seen https://github.com/aurghs/xarray-backend-tutorial/blob/main/2.Backend_with_Lazy_Loading.ipynb but it's a little bit different. While I have a custom backend array inheriting from `ZarrArrayWrapper`, this example using `ZarrArrayWrapper` directly still highlights the same unexpected behavior of everything being read in. ```python import zarr import xarray as xr from tempfile import mkdtemp import numpy as np from pathlib import Path from collections import defaultdict class AccessTrackingStore(zarr.DirectoryStore): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._access_count = {} self._accessed = defaultdict(set) def __getitem__(self, key): for tracked in self._access_count: if tracked in key: self._access_count[tracked] += 1 self._accessed[tracked].add(key) return super().__getitem__(key) def get_access_count(self, key): return self._access_count[key] def set_key_trackers(self, keys_to_track): if isinstance(keys_to_track, str): keys_to_track = [keys_to_track] for k in keys_to_track: self._access_count[k] = 0 def get_subkeys_accessed(self, key): return self._accessed[key] orig_path = Path(mkdtemp()) z = zarr.group(orig_path / ""foo.zarr"") z['array'] = np.random.randn(1000, 1000) store = AccessTrackingStore(orig_path / ""foo.zarr"") store.set_key_trackers(['array']) z = zarr.group(store) arr = xr.backends.zarr.ZarrArrayWrapper(z['array']) lazy_arr = xr.core.indexing.LazilyIndexedArray(arr) # just `.zarray` var = xr.Variable(('x', 'y'), lazy_arr) print('Variable read in ', store.get_subkeys_accessed('array')) # now everything is read in da = xr.DataArray(var) print('DataArray read in ', store.get_subkeys_accessed('array')) ```
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8753/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2135011804,I_kwDOAMm_X85_QbHc,8748,release v2024.02.0,2448579,closed,0,14808389,,0,2024-02-14T19:08:38Z,2024-02-18T22:52:15Z,2024-02-18T22:52:15Z,MEMBER,,,,"### What is your issue? Thanks to @keewis for volunteering at today's meeting :()","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8748/reactions"", ""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2086607437,I_kwDOAMm_X858XxpN,8616, new release 2024.01.0,2448579,closed,0,,,0,2024-01-17T17:03:20Z,2024-01-17T19:21:12Z,2024-01-17T19:21:12Z,MEMBER,,,,"### What is your issue? Thanks @TomNicholas for volunteering to drive this release!","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8616/reactions"", ""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2064420057,I_kwDOAMm_X857DIzZ,8581,bump min versions,2448579,closed,0,,,0,2024-01-03T17:45:10Z,2024-01-05T16:13:16Z,2024-01-05T16:13:15Z,MEMBER,,,,"### What is your issue? Looks like we can bump a number of min versions: ``` Package Required Policy Status ----------------- -------------------- -------------------- ------ cartopy 0.20 (2021-09-17) 0.21 (2022-09-10) < dask-core 2022.7 (2022-07-08) 2022.12 (2022-12-02) < distributed 2022.7 (2022-07-08) 2022.12 (2022-12-02) < flox 0.5 (2022-05-03) 0.6 (2022-10-12) < iris 3.2 (2022-02-15) 3.4 (2022-12-01) < matplotlib-base 3.5 (2021-11-18) 3.6 (2022-09-16) < numba 0.55 (2022-01-14) 0.56 (2022-09-28) < numpy 1.22 (2022-01-03) 1.23 (2022-06-23) < packaging 21.3 (2021-11-18) 22.0 (2022-12-08) < pandas 1.4 (2022-01-22) 1.5 (2022-09-19) < scipy 1.8 (2022-02-06) 1.9 (2022-07-30) < seaborn 0.11 (2020-09-08) 0.12 (2022-09-06) < typing_extensions 4.3 (2022-07-01) 4.4 (2022-10-07) < zarr 2.12 (2022-06-23) 2.13 (2022-09-27) < ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8581/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 2052952379,I_kwDOAMm_X856XZE7,8568,Raise when assigning attrs to virtual variables (default coordinate arrays),2448579,open,0,,,0,2023-12-21T19:24:11Z,2023-12-21T19:24:19Z,,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/8567
Originally posted by **matthew-brett** December 21, 2023 Sorry for the introductory question, but we (@ivanov and I) ran into this behavior while experimenting: ```python import numpy as np data = np.zeros((3, 4, 5)) ds = xr.DataArray(data, dims=('i', 'j', 'k')) print(ds['k'].attrs) ``` This shows `{}` as we might reasonably expect. But then: ```python ds['k'].attrs['foo'] = 'bar' print(ds['k'].attrs) ``` This also gives `{}`, which we found surprising. We worked out why that was, after a little experimentation (the default coordinate arrays seems to get created on the fly and garbage collected immediately). But it took us a little while. Is that as intended? Is there a way of making this less confusing? Thanks for any help.
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8568/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1954809370,I_kwDOAMm_X850hAYa,8353,Update benchmark suite for asv 0.6.1,2448579,open,0,,,0,2023-10-20T18:13:22Z,2023-12-19T05:53:21Z,,MEMBER,,,,"The new asv version comes with decorators for parameterizing and skipping, and the ability to use `mamba` to create environments. https://github.com/airspeed-velocity/asv/releases https://asv.readthedocs.io/en/v0.6.1/writing_benchmarks.html#skipping-benchmarks This might help us reduce benchmark times a bit, or at least simplify the code some. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8353/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1989588884,I_kwDOAMm_X852lreU,8448,mypy 1.7.0 raising errors,2448579,closed,0,,,0,2023-11-12T21:41:43Z,2023-12-01T22:02:22Z,2023-12-01T22:02:22Z,MEMBER,,,,"### What happened? ``` xarray/namedarray/core.py:758: error: Value of type Never is not indexable [index] xarray/core/alignment.py:684: error: Unused ""type: ignore"" comment [unused-ignore] xarray/core/alignment.py:1156: error: Unused ""type: ignore"" comment [unused-ignore] xarray/core/dataset.py: note: In member ""sortby"" of class ""Dataset"": xarray/core/dataset.py:7967: error: Incompatible types in assignment (expression has type ""tuple[Alignable, ...]"", variable has type ""tuple[DataArray, ...]"") [assignment] xarray/core/dataset.py:7979: error: ""Alignable"" has no attribute ""isel"" [attr-defined] ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8448/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1942893480,I_kwDOAMm_X85zzjOo,8306,keep_attrs for NamedArray,2448579,open,0,,,0,2023-10-14T02:29:54Z,2023-10-14T02:31:35Z,,MEMBER,,,,"### What is your issue? Copying over @max-sixty's comment from https://github.com/pydata/xarray/pull/8304#discussion_r1358873522 > I haven't been in touch with the NameArray discussions so forgive a glib comment — but re https://github.com/pydata/xarray/issues/3891 — this would be a ""once-in-a-library"" opportunity to always retain attrs in aggregations, removing the `keep_attrs` option in methods. > > (Xarray could still handle them as it wished, so xarray's external interface wouldn't need to change immediately...) @pydata/xarray Should we just delete the `keep_attrs` kwarg completely for NamedArray and always propagate attrs? `obj.attrs.clear()` seems just as easy to type.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8306/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1888576440,I_kwDOAMm_X85wkWO4,8162,Update group by multi index,2448579,open,0,,,0,2023-09-09T04:50:29Z,2023-09-09T04:50:39Z,,MEMBER,,,,"ideally `GroupBy._infer_concat_args()` would return a `xr.Coordinates` object that contains both the coordinate(s) and their (multi-)index to assign to the result (combined) object. The goal is to avoid calling `create_default_index_implicit(coord)` below where `coord` is a `pd.MultiIndex` or a single `IndexVariable` wrapping a multi-index. If `coord` is a `Coordinates` object, we could do `combined = combined.assign_coords(coord)` instead. https://github.com/pydata/xarray/blob/e2b6f3468ef829b8a83637965d34a164bf3bca78/xarray/core/groupby.py#L1573-L1587 There are actually more general issues: - The `group` parameter of Dataset.groupby being a single variable or variable name, it won't be possible to do groupby on a full pandas multi-index once we drop its dimension coordinate (#8143). How can we still support it? Maybe passing a dimension name to `group` and check that there's only one index for that dimension? - How can we support custom, multi-coordinate indexes with groupby? I don't have any practical example in mind, but in theory just passing a single coordinate name as `group` will invalidate the index. Should we drop the index in the result? Or, like suggested above pass a dimension name as group and check the index? _Originally posted by @benbovy in https://github.com/pydata/xarray/issues/8140#issuecomment-1709775666_ ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1812504689,I_kwDOAMm_X85sCKBx,8006,Fix documentation about datetime_unit of xarray.DataArray.differentiate,2448579,closed,0,,,0,2023-07-19T18:31:10Z,2023-09-01T09:37:15Z,2023-09-01T09:37:15Z,MEMBER,,,,"Should say that `Y` and `M` cannot be supported with `datetime64` ### Discussed in https://github.com/pydata/xarray/discussions/8000
Originally posted by **jesieleo** July 19, 2023 I have a piece of data that looks like this ``` Dimensions: (time: 612, LEV: 15, latitude: 20, longitude: 357) Coordinates: * time (time) datetime64[ns] 1960-01-15 1960-02-15 ... 2010-12-15 * LEV (LEV) float64 5.01 15.07 25.28 35.76 ... 149.0 171.4 197.8 229.5 * latitude (latitude) float64 -4.75 -4.25 -3.75 -3.25 ... 3.75 4.25 4.75 * longitude (longitude) float64 114.2 114.8 115.2 115.8 ... 291.2 291.8 292.2 Data variables: u (time, LEV, latitude, longitude) float32 ... Attributes: (12/30) cdm_data_type: Grid Conventions: COARDS, CF-1.6, ACDD-1.3 creator_email: chepurin@umd.edu creator_name: APDRC creator_type: institution creator_url: https://www.atmos.umd.edu/~ocean/ ... ... standard_name_vocabulary: CF Standard Name Table v29 summary: Simple Ocean Data Assimilation (SODA) soda po... time_coverage_end: 2010-12-15T00:00:00Z time_coverage_start: 1983-01-15T00:00:00Z title: SODA soda pop2.2.4 [TIME][LEV][LAT][LON] Westernmost_Easting: 118.25 ``` when i try to use xarray.DataArray.differentiate `data.u.differentiate('time',datetime_unit='M')` will appear ``` Traceback (most recent call last): File """", line 1, in File ""D:\Anaconda3\lib\site-packages\xarray\core\dataarray.py"", line 3609, in differentiate ds = self._to_temp_dataset().differentiate(coord, edge_order, datetime_unit) File ""D:\Anaconda3\lib\site-packages\xarray\core\dataset.py"", line 6372, in differentiate coord_var = coord_var._to_numeric(datetime_unit=datetime_unit) File ""D:\Anaconda3\lib\site-packages\xarray\core\variable.py"", line 2428, in _to_numeric numeric_array = duck_array_ops.datetime_to_numeric( File ""D:\Anaconda3\lib\site-packages\xarray\core\duck_array_ops.py"", line 466, in datetime_to_numeric array = array / np.timedelta64(1, datetime_unit) TypeError: Cannot get a common metadata divisor for Numpy datatime metadata [ns] and [M] because they have incompatible nonlinear base time units. ``` Would you please told me is this a BUG?
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8006/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1824824446,I_kwDOAMm_X85sxJx-,8025,"Support Groupby first, last with flox",2448579,open,0,,,0,2023-07-27T17:07:51Z,2023-07-27T19:08:06Z,,MEMBER,,,,"### Is your feature request related to a problem? [flox](https://github.com/xarray-contrib/flox) recently added support for first, last, nanfirst, nanlast. So we should support that on the Xarray GroupBy object. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8025/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1642299599,I_kwDOAMm_X85h44DP,7683,automatically chunk in groupby binary ops,2448579,closed,0,,,0,2023-03-27T15:14:09Z,2023-07-27T16:41:35Z,2023-07-27T16:41:34Z,MEMBER,,,,"### What happened? From https://discourse.pangeo.io/t/xarray-unable-to-allocate-memory-how-to-size-up-problem/3233/4 Consider ``` python # ds is dataset with big dask arrays mean = ds.groupby(""time.day"").mean() mean.to_netcdf() mean = xr.open_dataset(...) ds.groupby(""time.day"") - mean ``` In `GroupBy._binary_op` https://github.com/pydata/xarray/blob/39caafae4452f5327a7cd671b18d4bb3eb3785ba/xarray/core/groupby.py#L616 we will eagerly construct `other` that is of the same size as `ds`. ### What did you expect to happen? I think the only solution is to automatically chunk if `ds` has dask arrays, and `other` (or `mean`) isn't backed by dask arrays. A chunk size of `1` seems sensible. ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7683/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1797636782,I_kwDOAMm_X85rJcKu,7976,Explore updating colormap code,2448579,closed,0,,,0,2023-07-10T21:51:30Z,2023-07-11T13:49:54Z,2023-07-11T13:49:53Z,MEMBER,,,,"### What is your issue? See https://github.com/matplotlib/matplotlib/issues/16296 Looks like the MPL API may have advanced enough that we can delete some of our use of private attributes.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7976/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1692597701,I_kwDOAMm_X85k4v3F,7808,Default behaviour of `min_count` wrong with flox,2448579,closed,0,,,0,2023-05-02T15:04:11Z,2023-05-10T02:39:45Z,2023-05-10T02:39:45Z,MEMBER,,,,"### What happened? ```python with xr.set_options(display_style=""text"", use_flox=False): with xr.set_options(use_flox=False): display( xr.DataArray( data=np.array([np.nan, 1, 1, np.nan, 1, 1]), dims=""x"", coords={""labels"": (""x"", np.array([1, 2, 3, 1, 2, 3]))}, ) .groupby(""labels"") .sum() ) with xr.set_options(use_flox=True): display( xr.DataArray( data=np.array([np.nan, 1, 1, np.nan, 1, 1]), dims=""x"", coords={""labels"": (""x"", np.array([1, 2, 3, 1, 2, 3]))}, ) .groupby(""labels"") .sum() ) ``` ``` # without flox array([0., 2., 2.]) Coordinates: * labels (labels) int64 1 2 3 # with flox array([nan, 2., 2.]) Coordinates: * labels (labels) int64 1 2 3 ``` ### What did you expect to happen? The same answer. We should set `min_count=0` when `min_count is None` ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7808/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1649611456,I_kwDOAMm_X85iUxLA,7704,follow upstream scipy interpolation improvements,2448579,open,0,,,0,2023-03-31T15:46:56Z,2023-03-31T15:46:56Z,,MEMBER,,,,"### Is your feature request related to a problem? Scipy 1.10.0 has some great improvements to interpolation ([release notes](https://docs.scipy.org/doc/scipy/release.1.10.0.html#scipy-interpolate-improvements)) particularly around the fancier methods like `pchip`. It'd be good to see if we can simplify some of our code (or even enable using these options). ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7704/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 626591460,MDU6SXNzdWU2MjY1OTE0NjA=,4107,renaming Variable to a dimension name does not convert to IndexVariable,2448579,closed,0,4160723,,0,2020-05-28T15:11:49Z,2022-09-27T09:33:42Z,2022-09-27T09:33:42Z,MEMBER,,,," Seen in #4103 #### MCVE Code Sample ```python from xarray.tests import assert_identical coord_1 = xr.DataArray([1, 2], dims=[""coord_1""], attrs={""attrs"": True}) da = xr.DataArray([1, 0], [coord_1]) obj = da.reset_index(""coord_1"").rename({""coord_1_"": ""coord_1""}) assert_identical(da, obj) ``` #### Expected Output #### Problem Description ``` AssertionErrorTraceback (most recent call last) in ----> 1 assert_identical(da, obj) ~/work/python/xarray/xarray/tests/__init__.py in assert_identical(a, b) 160 xarray.testing.assert_identical(a, b) 161 xarray.testing._assert_internal_invariants(a) --> 162 xarray.testing._assert_internal_invariants(b) 163 164 ~/work/python/xarray/xarray/testing.py in _assert_internal_invariants(xarray_obj) 265 _assert_variable_invariants(xarray_obj) 266 elif isinstance(xarray_obj, DataArray): --> 267 _assert_dataarray_invariants(xarray_obj) 268 elif isinstance(xarray_obj, Dataset): 269 _assert_dataset_invariants(xarray_obj) ~/work/python/xarray/xarray/testing.py in _assert_dataarray_invariants(da) 210 assert all( 211 isinstance(v, IndexVariable) for (k, v) in da._coords.items() if v.dims == (k,) --> 212 ), {k: type(v) for k, v in da._coords.items()} 213 for k, v in da._coords.items(): 214 _assert_variable_invariants(v, k) AssertionError: {'coord_1': } ``` #### Versions
Output of xr.show_versions()
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4107/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1378174355,I_kwDOAMm_X85SJUWT,7055,Use roundtrip context manager in distributed write tests,2448579,open,0,,,0,2022-09-19T15:53:40Z,2022-09-19T15:53:40Z,,MEMBER,,,,"### What is your issue? File roundtripping tests in `test_distributed.py` don't use the `roundtrip` context manager (thpugh one uses `create_tmp_file`) so I don't think any created files are being cleaned up. Example: https://github.com/pydata/xarray/blob/09e467a6a3a8ed68c6c29647ebf2b09288145da1/xarray/tests/test_distributed.py#L91-L119","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7055/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1289174987,I_kwDOAMm_X85M1z_L,6739,"""center"" kwarg ignored when manually iterating over DataArrayRolling",2448579,closed,0,,,0,2022-06-29T19:07:07Z,2022-07-14T17:41:01Z,2022-07-14T17:41:01Z,MEMBER,,,,"### Discussed in https://github.com/pydata/xarray/discussions/6738
Originally posted by **ckingdon95** June 29, 2022 Hello, I am trying to manually iterate over a DataArrayRolling object, as described [here ](https://docs.xarray.dev/en/stable/user-guide/computation.html#rolling-window-operations)in the documentation. I am confused why the following two code chunks do not produce the same sequence of values. I would like to be able to manually iterate over a DataArrayRolling object, and still be given center-justified windows. Is there a way to do this? ```python import xarray as xr import numpy as np my_data = xr.DataArray(np.arange(1,10), dims=""x"") # Option 1: take a center-justified rolling average result1 = my_data.rolling(x=3, center=True).mean().values result1 ``` This returns the following values, as expected: ``` array([nan, 2., 3., 4., 5., 6., 7., 8., nan]) ``` Whereas when I do it manually, it is not equivalent: ```python # Option 2: try to manually iterate, but the result is not centered my_data_rolling = my_data.rolling(x=3, center=True) result2 = [window.mean().values.item() for label, window in my_data_rolling] result2 ``` This returns ``` [nan, nan, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0] ``` Is this an issue with the window iterator? If it is not an issue, then is there a way for me to get the center-justified windows in the manual iteration?
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6739/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 968977385,MDU6SXNzdWU5Njg5NzczODU=,5699,describe options in documentation,2448579,closed,0,,,0,2021-08-12T14:48:00Z,2022-06-25T20:01:07Z,2022-06-25T20:01:07Z,MEMBER,,,,"I think we only describe available options in the API reference for `xr.set_options` It'd be nice to add a ""Configuring Xarray"" section in the User Guide. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5699/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1178907807,I_kwDOAMm_X85GRLSf,6407,Add backend tutorial material,2448579,closed,0,,,0,2022-03-24T03:44:22Z,2022-06-23T01:51:44Z,2022-06-23T01:51:44Z,MEMBER,,,,"### What is your issue? @aurghs developed some nice backend tutorial material for the Dask Summit: https://github.com/aurghs/xarray-backend-tutorial It'd be nice to add it either to our main documentation or to https://github.com/xarray-contrib/xarray-tutorial.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1238783899,I_kwDOAMm_X85J1leb,6616,flox breaks multiindex groupby,2448579,closed,0,,,0,2022-05-17T15:05:00Z,2022-05-17T16:11:18Z,2022-05-17T16:11:18Z,MEMBER,,,,"### What happened? From @malmans2 ``` python import numpy as np import xarray as xr ds = xr.Dataset( dict(a=((""z"",), np.ones(10))), coords=dict(b=((""z""), np.arange(2).repeat(5)), c=((""z""), np.arange(5).repeat(2))), ).set_index(bc=[""b"", ""c""]) grouped = ds.groupby(""bc"") with xr.set_options(use_flox=False): grouped.sum() # OK with xr.set_options(use_flox=True): grouped.sum() # Error ``` ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output ```Python ctests/test_xarray.py:329: in test_multi_index_groupby_sum actual = xarray_reduce(ds, ""bc"", func=""sum"") flox/xarray.py:374: in xarray_reduce actual[k] = v.expand_dims(missing_group_dims) ../xarray/xarray/core/dataset.py:1427: in __setitem__ self.update({key: value}) ../xarray/xarray/core/dataset.py:4432: in update merge_result = dataset_update_method(self, other) ../xarray/xarray/core/merge.py:1070: in dataset_update_method return merge_core( ../xarray/xarray/core/merge.py:722: in merge_core aligned = deep_align( ../xarray/xarray/core/alignment.py:824: in deep_align aligned = align( ../xarray/xarray/core/alignment.py:761: in align aligner.align() ../xarray/xarray/core/alignment.py:550: in align self.assert_unindexed_dim_sizes_equal() ../xarray/xarray/core/alignment.py:450: in assert_unindexed_dim_sizes_equal raise ValueError( E ValueError: cannot reindex or align along dimension 'bc' because of conflicting dimension sizes: {10, 6} (note: an index is found along that dimension with size=10) ______________________________ test_multi_index_groupby_sum[numpy] _______________________________ tests/test_xarray.py:329: in test_multi_index_groupby_sum actual = xarray_reduce(ds, ""bc"", func=""sum"") flox/xarray.py:374: in xarray_reduce actual[k] = v.expand_dims(missing_group_dims) ../xarray/xarray/core/dataset.py:1427: in __setitem__ self.update({key: value}) ../xarray/xarray/core/dataset.py:4432: in update merge_result = dataset_update_method(self, other) ../xarray/xarray/core/merge.py:1070: in dataset_update_method return merge_core( ../xarray/xarray/core/merge.py:722: in merge_core aligned = deep_align( ../xarray/xarray/core/alignment.py:824: in deep_align aligned = align( ../xarray/xarray/core/alignment.py:761: in align aligner.align() ../xarray/xarray/core/alignment.py:550: in align self.assert_unindexed_dim_sizes_equal() ../xarray/xarray/core/alignment.py:450: in assert_unindexed_dim_sizes_equal raise ValueError( E ValueError: cannot reindex or align along dimension 'bc' because of conflicting dimension sizes: {10, 6} (note: an index is found along that dimension with size=10) ``` ``` ### Anything else we need to know? _No response_ ### Environment
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6616/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1235494254,I_kwDOAMm_X85JpCVu,6606,Fix benchmark CI,2448579,closed,0,,,0,2022-05-13T17:18:32Z,2022-05-14T23:06:44Z,2022-05-14T23:06:44Z,MEMBER,,,,"### What is your issue? It's failing during setup: https://github.com/pydata/xarray/runs/6424624397?check_suite_focus=true ``` · Discovering benchmarks ·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Building dd20d07f for conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/bin/python -mpip wheel --no-deps --no-index -w /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/asv-build-cache/dd20d07f4057a9e29222ca132c36cbaaf3fbb242 /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project (exit status 1) STDOUT --------> Processing /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project STDERR --------> ERROR: Some build dependencies for file:///home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project are missing: 'setuptools_scm[toml]>=3.4', 'setuptools_scm_git_archive'. ·· Failed: trying different commit/environment ·· Uninstalling from conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Building c34ef8a6 for conda-py3.8-bottleneck-dask-distributed-flox-netcdf4-numpy-numpy_groupies-pandas-scipy-sparse ·· Error running /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/bin/python -mpip wheel --no-deps --no-index -w /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/asv-build-cache/c34ef8a60227720724e90aa11a6266c0026a812a /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project (exit status 1) STDOUT --------> Processing /home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project STDERR --------> ERROR: Some build dependencies for file:///home/runner/work/xarray/xarray/asv_bench/.asv/env/e8ce5703538597037a298414451d04d2/project are missing: 'setuptools_scm[toml]>=3.4', 'setuptools_scm_git_archive'. ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1180334986,I_kwDOAMm_X85GWnuK,6411,Better dask support in polyval,2448579,closed,0,,,0,2022-03-25T04:35:48Z,2022-05-05T20:17:07Z,2022-05-05T20:17:07Z,MEMBER,,,,"### Is your feature request related to a problem? polyval does not handle dask inputs well. ```python nt = 8772 // 4 ny = 489 nx = 655 # chunks like the data is stored on disk # small in time, big in space # because the chunk sizes are -1 along lat, lon; # reshaping this array to (time, latlon) prior to fitting is pretty cheap chunks = (8, -1, -1) da = xr.DataArray( dask.array.random.random((nt, ny, nx), chunks=chunks), dims=(""ocean_time"", ""eta_rho"", ""xi_rho""), ) dim = ""ocean_time"" deg = 1 p = da.polyfit(dim=""ocean_time"", deg=1, skipna=False) # create a chunked version of the ""ocean_time"" dimension chunked_dim = xr.DataArray( dask.array.from_array(da[dim].data, chunks=da.chunksizes[dim]), dims=dim, name=dim ) xr.polyval(chunked_dim, p.polyfit_coefficients) ``` ### Describe the solution you'd like Here's a partial solution. It does not handle datetime inputs (polyval handles this using `get_clean_interp_index` which computes dask inputs). But I've replaced the call to `np.vander` and used `xr.dot`. ```python def polyval(coord, coeffs, degree_dim=""degree""): x = coord.data deg_coord = coeffs[degree_dim] N = int(deg_coord.max()) + 1 lhs = xr.DataArray( np.stack([x ** (N - 1 - i) for i in range(N)], axis=1), dims=(coord.name, degree_dim), coords={coord.name: coord, degree_dim: np.arange(deg_coord.max() + 1)[::-1]}, ) return xr.dot(lhs, coeffs, dims=degree_dim) polyval(chunked_dim, p.polyfit_coefficients) ``` This looks like what I expected cc @aulemahal ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6411/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1203414243,I_kwDOAMm_X85HuqTj,6481,refactor broadcast for flexible indexes,2448579,open,0,,,0,2022-04-13T14:51:19Z,2022-04-13T14:51:28Z,,MEMBER,,,,"### What is your issue? From @benbovy in https://github.com/pydata/xarray/pull/6477 > - extract common indexes and explicitly pass them to the Dataset and DataArray constructors (when implemented) that are called in the broadcast helper functions (there are some temporary and ugly hacks in create_default_index_implicit so that it works now with pandas multi-indexes wrapped in coordinate variables without the need to pass those indexes explicitly) > - extract common indexes based on the dimension(s) of their coordinates and not their name (e.g., case of non-dimension but indexed coordinate) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6481/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1194790343,I_kwDOAMm_X85HNw3H,6445,map removes non-dimensional coordinate variables,2448579,open,0,,,0,2022-04-06T15:40:40Z,2022-04-06T15:40:40Z,,MEMBER,,,,"### What happened? ``` python ds = xr.Dataset( {""a"": (""x"", [1, 2, 3])}, coords={""c"": (""x"", [1, 2, 3]), ""d"": (""y"", [1, 2, 3, 4])} ) print(ds.coords) mapped = ds.map(lambda x: x) print(mapped.coords) ``` Variables `d` gets dropped in the `map` call. It does not share any dimensions with any of the data variables. ``` Coordinates: c (x) int64 1 2 3 d (y) int64 1 2 3 4 Coordinates: c (x) int64 1 2 3 ``` ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment xarray 2022.03.0","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6445/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1171916710,I_kwDOAMm_X85F2gem,6372,"apply_ufunc + dask=""parallelized"" + no core dimensions should raise a nicer error about core dimensions being absent",2448579,open,0,,,0,2022-03-17T04:25:37Z,2022-03-17T05:10:16Z,,MEMBER,,,,"### What happened? From https://github.com/pydata/xarray/discussions/6370 Calling `apply_ufunc(..., dask=""parallelized"")` with no core dimensions and dask input ""works"" but raises an error on compute (`ValueError: axes don't match array` from `np.transpose`). ``` python xr.apply_ufunc( lambda x: np.mean(x), dt, dask=""parallelized"" ) ``` ### What did you expect to happen? With numpy data the apply_ufunc call does raise an error: ``` xr.apply_ufunc( lambda x: np.mean(x), dt.compute(), dask=""parallelized"" ) ``` ``` ValueError: applied function returned data with unexpected number of dimensions. Received 0 dimension(s) but expected 1 dimensions with names: ('x',) ``` ### Minimal Complete Verifiable Example ``` python import xarray as xr dt = xr.Dataset( data_vars=dict( value=([""x""], [1,1,2,2,2,3,3,3,3,3]), ), coords=dict( lon=([""x""], np.linspace(0,1,10)), ), ).chunk(chunks={'x': tuple([2,3,5])}) # three chunks of different size xr.apply_ufunc( lambda x: np.mean(x), dt, dask=""parallelized"" ) ``` ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment N/A","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6372/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1048856436,I_kwDOAMm_X84-hEd0,5962,Test resampling with dask arrays,2448579,open,0,,,0,2021-11-09T17:02:45Z,2021-11-09T17:02:45Z,,MEMBER,,,,"I noticed that we don't test resampling with dask arrays (well just one). This could be a good opportunity to convert `test_groupby.py` to use test fixtures like in https://github.com/pydata/xarray/pull/5411 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5962/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1043846371,I_kwDOAMm_X84-N9Tj,5934,add test for custom backend entrypoint,2448579,open,0,,,0,2021-11-03T16:57:14Z,2021-11-03T16:57:21Z,,MEMBER,,,,"From https://github.com/pydata/xarray/pull/5931 It would be good to add a test checking that custom backend entrypoints work. This might involve creating a dummy package that registers an entrypoint (https://github.com/pydata/xarray/pull/5931#issuecomment-959131968) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 938141608,MDU6SXNzdWU5MzgxNDE2MDg=,5582,Faster unstacking of dask arrays,2448579,open,0,,,0,2021-07-06T18:12:05Z,2021-07-06T18:54:40Z,,MEMBER,,,,"Recent dask version support assigning to a list of ints along one dimension. we can use this for unstacking (diff builds on #5577) ```diff diff --git i/xarray/core/variable.py w/xarray/core/variable.py index 222e8dab9..a50dfc574 100644 --- i/xarray/core/variable.py +++ w/xarray/core/variable.py @@ -1593,11 +1593,9 @@ class Variable(AbstractArray, NdimSizeLenMixin, VariableArithmetic): else: dtype = self.dtype - if sparse: + if sparse and not is_duck_dask_array(reordered): # unstacking a dense multitindexed array to a sparse array - # Use the sparse.COO constructor until sparse supports advanced indexing - # https://github.com/pydata/sparse/issues/114 - # TODO: how do we allow different sparse array types + # Use the sparse.COO constructor since we cannot assign to sparse.COO from sparse import COO codes = zip(*index.codes) @@ -1618,19 +1616,23 @@ class Variable(AbstractArray, NdimSizeLenMixin, VariableArithmetic): ) else: + # dask supports assigning to a list of ints along one axis only. + # So we construct an array with the last dimension flattened, + # assign the values, then reshape to the final shape. + intermediate_shape = reordered.shape[:-1] + (np.prod(new_dim_sizes),) + indexer = np.ravel_multi_index(index.codes, new_dim_sizes) data = np.full_like( self.data, fill_value=fill_value, - shape=new_shape, + shape=intermediate_shape, dtype=dtype, ) # Indexer is a list of lists of locations. Each list is the locations # on the new dimension. This is robust to the data being sparse; in that # case the destinations will be NaN / zero. - # sparse doesn't support item assigment, - # https://github.com/pydata/sparse/issues/114 - data[(..., *indexer)] = reordered + data[(..., indexer)] = reordered + data = data.reshape(new_shape) return self._replace(dims=new_dims, data=data) ``` This should be what `alignment.reindex_variables` is doing but I don't fully understand that function. The annoying bit is figuring out when to use this version and what to do with things like dask wrapping sparse. I think we want to loop over each variable in `Dataset.unstack` calling `Variable.unstack` and dispatch based on the type of `Variable.data` to easily handle all the edge cases. cc @Illviljan if you're interested in implementing this","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5582/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 798586325,MDU6SXNzdWU3OTg1ODYzMjU=,4852,mention HDF files in docs,2448579,open,0,,,0,2021-02-01T18:05:23Z,2021-07-04T01:24:22Z,,MEMBER,,,,"This is such a common question that we should address it in the docs. Just saying that some hdf5 files can be opened with `h5netcdf`, and that the user needs to manually create xarray objects with everything else should be enough. https://xarray.pydata.org/en/stable/io.html","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4852/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 664595680,MDU6SXNzdWU2NjQ1OTU2ODA=,4260,use matplotlib's categorical axis features,2448579,closed,0,,,0,2020-07-23T16:01:13Z,2021-06-21T17:45:39Z,2021-06-21T17:45:39Z,MEMBER,,,," **Is your feature request related to a problem? Please describe.** xarray currently doesn't allow plotting against coordinates with string labels for example. **Describe the solution you'd like** Use matplotlib's categorical axis support. Example: https://matplotlib.org/gallery/lines_bars_and_markers/categorical_variables.html This may be the only place a change is required: https://github.com/pydata/xarray/blob/4e893317240ed1a80e65ea2de107e9179bb65446/xarray/plot/utils.py#L572-L608 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4260/reactions"", ""total_count"": 8, ""+1"": 8, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 685590739,MDU6SXNzdWU2ODU1OTA3Mzk=,4373,Add Dataset.plot.quiver,2448579,closed,0,,,0,2020-08-25T15:39:37Z,2021-02-19T14:21:45Z,2021-02-19T14:21:45Z,MEMBER,,,,"I think it would be nice to add a quiver plot function. I got this far in my current project: ``` python @xarray.plot.dataset_plot._dsplot def quiver(ds, x, y, ax, u, v, **kwargs): from xarray import broadcast if x is None or y is None or u is None or v is None: raise ValueError(""Must specify x, y, u, v for quiver plots."") # matplotlib autoscaling algorithm scale = kwargs.pop(""scale"", None) if scale is None: npts = ds.dims[x] * ds.dims[y] # crude auto-scaling # scale is typical arrow length as a multiple of the arrow width scale = ( 1.8 * ds.to_array().median().values * np.maximum(10, np.sqrt(npts)) ) # / span ds = ds.squeeze() x, y, u, v = broadcast(ds[x], ds[y], ds[u], ds[v]) # no support for these kwargs right now. kwargs.pop(""cmap_params"") kwargs.pop(""hue"") kwargs.pop(""hue_style"") hdl = ax.quiver(x.values, y.values, u.values, v.values, scale=scale, **kwargs) return hdl ``` The autoscaling logic is quite crude; I tried to copy what matplotlib does but got somewhat confused. To get faceting to work properly, we'll need to estimate `scale` around here: https://github.com/pydata/xarray/blob/a36d0a1d4657c848dcdd76d0ecb9c783ad464057/xarray/plot/facetgrid.py#L339-L343","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4373/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 797053785,MDU6SXNzdWU3OTcwNTM3ODU=,4848,simplify API reference presentation,2448579,open,0,,,0,2021-01-29T17:23:41Z,2021-01-29T17:23:46Z,,MEMBER,,,,"Can we remove `xarray.core.rolling` and `core.rolling` on the left and right respectively? I think the API reference would be a lot more readable if we could do that ![image](https://user-images.githubusercontent.com/2448579/106306900-e7f27e00-621b-11eb-8ddd-2b9c15abdfca.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4848/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 787486472,MDU6SXNzdWU3ODc0ODY0NzI=,4817,Add encoding to HTML repr,2448579,open,0,,,0,2021-01-16T15:14:50Z,2021-01-24T17:31:31Z,,MEMBER,,,," **Is your feature request related to a problem? Please describe.** `.encoding` is somewhat hidden since we don't show it in a repr. **Describe the solution you'd like** I think it'd be nice to add it to the HTML repr, collapsed by default. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4817/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 484955475,MDU6SXNzdWU0ODQ5NTU0NzU=,3263,test_sparse doesn't work with pytest-xdist,2448579,closed,0,,,0,2019-08-25T17:30:51Z,2020-12-17T22:33:46Z,2020-12-17T22:33:46Z,MEMBER,,,,"`pytest -n 4 test_sparse.py` results in ``` ====================================================================================== ERRORS ====================================================================================== _______________________________________________________________________________ ERROR collecting gw0 _______________________________________________________________________________ Different tests were collected between gw1 and gw0. The difference is: --- gw1 +++ gw0 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] _______________________________________________________________________________ ERROR collecting gw2 _______________________________________________________________________________ Different tests were collected between gw1 and gw2. The difference is: --- gw1 +++ gw2 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -118,7 +118,7 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2]})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.shift(*(), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sortby(*('x',), **{'ascending': False})-True] -xarray/tests/test_sparse.py::test_dataarray_method[obj.stack(*(), **{'z': {'y', 'x'}})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.stack(*(), **{'z': {'x', 'y'}})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.transpose(*(), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.broadcast_equals(*(\n,), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.equals(*(\n,), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] _______________________________________________________________________________ ERROR collecting gw3 _______________________________________________________________________________ Different tests were collected between gw1 and gw3. The difference is: --- gw1 +++ gw3 @@ -11,7 +11,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.any(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.astype(*(), **{'dtype': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.clip(*(), **{'min': 0, 'max': 1})-True] -xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] +xarray/tests/test_sparse.py::test_variable_method[obj.coarsen(*(), **{'windows': {'x': 2}, 'func': })-True] xarray/tests/test_sparse.py::test_variable_method[obj.compute(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.conj(*(), **{})-True] xarray/tests/test_sparse.py::test_variable_method[obj.copy(*(), **{})-True] @@ -49,7 +49,7 @@ xarray/tests/test_sparse.py::test_variable_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_variable_method[obj.quantile(*(), **{'q': 0.5})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rank(*(), **{'dim': 'x'})-False] -xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] +xarray/tests/test_sparse.py::test_variable_method[obj.reduce(*(), **{'func': , 'dim': 'x'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.rolling_window(*(), **{'dim': 'x', 'window': 2, 'window_dim': 'x_win'})-True] xarray/tests/test_sparse.py::test_variable_method[obj.shift(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_variable_method[obj.std(*(), **{})-False] @@ -144,11 +144,11 @@ xarray/tests/test_sparse.py::test_dataarray_method[obj.median(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.min(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.notnull(*(), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] +xarray/tests/test_sparse.py::test_dataarray_method[obj.pipe(*(,), **{'axis': 1})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.prod(*(), **{})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.quantile(*(), **{'q': 0.5})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.rank(*('x',), **{})-False] -xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] +xarray/tests/test_sparse.py::test_dataarray_method[obj.reduce(*(,), **{'dim': 'x'})-False] xarray/tests/test_sparse.py::test_dataarray_method[obj.reindex_like(*(\n\nCoordinates:\n * x (x) float64 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5\n * y (y) float64 0.5 1.5 2.5 3.5 4.5,), **{})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.roll(*(), **{'x': 2})-True] xarray/tests/test_sparse.py::test_dataarray_method[obj.sel(*(), **{'x': [0, 1, 2], 'y': [2, 3]})-True] ================================================================================= warnings summary ================================================================================= /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/client.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import defaultdict, Iterator /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/publish.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2 /home/deepak/miniconda3/envs/dcpy/lib/python3.7/site-packages/distributed/scheduler.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import defaultdict, deque, OrderedDict, Mapping, Set -- Docs: https://docs.pytest.org/en/latest/warnings.html ======================================================================= ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3263/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 648250671,MDU6SXNzdWU2NDgyNTA2NzE=,4189,List supported options for `backend_kwargs` in `open_dataset`,2448579,open,0,,,0,2020-06-30T15:01:31Z,2020-12-15T04:28:04Z,,MEMBER,,,,"We should list supported options for `backend_kwargs` in the docstring for `open_dataset`and possibly in `io.rst` xref #4187 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4189/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 526754376,MDU6SXNzdWU1MjY3NTQzNzY=,3558,optimizing xarray operations for lazy array equality test,2448579,closed,0,,,0,2019-11-21T18:01:51Z,2020-02-24T18:26:30Z,2020-02-24T18:26:30Z,MEMBER,,,,"TLDR: I think we want `A.sel(x=A.x).equals(A)` to pass lazily. It doesn't do so currently. Currently if I do `A.sel(x=A.x)`, this sticks in a `getitem` call in the dask graph which breaks our lazy array equality optimization. Here's an example ``` python >>> A = xr.DataArray(np.arange(100), dims=""x"", coords={""x"": np.arange(100)}).chunk({""x"": 1}) >>> A.sel(x=A.x).variable.equals(A, equiv=xr.core.duck_array_ops.lazy_array_equiv) None ``` **Questions:** 1. Where is the best place to do this? In `sel` or `isel`? Both? Sticking the following in `sel` makes the above check return `True` which is what we want: ``` python if self._indexes: equals = [] for index in indexers: equals.append(indexers[index].to_index().equals(self._indexes[index])) if all(equals): return self ``` This doesn't handle slice objects though so that makes me think we'd want to add something similar to `isel` too. 2. What is the behaviour we want? `A.sel(x=A.x).equals(A)` or `A.sel(x=A.x) is A`? 3. Doing the latter will mean changing `_to_temp_dataset` and `_from_temp_dataset` which suggests the constraint `A._from_temp_dataset(A._to_temp_dataset()) is A`? But this seems too strong to me. Do we only want to lazily satisfy an `equals` constraint rather than an `identical` constraint? 4. It seems like we'll want to add such short-circuits in many places (I have not checked all of these): `sortby`, `broadcast`, `align`, `reindex` (`transpose` does this now).","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3558/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 556998262,MDU6SXNzdWU1NTY5OTgyNjI=,3725,pip warning on CI,2448579,closed,0,,,0,2020-01-29T17:07:49Z,2020-01-29T23:39:39Z,2020-01-29T23:39:39Z,MEMBER,,,,"We're getting this warning on CI (e.g. https://dev.azure.com/xarray/xarray/_build/results?buildId=2047&view=logs&j=e9c23135-6f4c-5980-91c2-81d28ce70c9b&t=bd167cdf-93c0-5e40-5b11-b5faeb5dc22f) ``` WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. ``` cc @crusaderky ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3725/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 538521723,MDU6SXNzdWU1Mzg1MjE3MjM=,3630,reviewnb for example notebooks?,2448579,open,0,,,0,2019-12-16T16:34:28Z,2019-12-16T16:34:28Z,,MEMBER,,,,"What do people think of adding ReviewNB https://www.reviewnb.com/ to facilitate easy reviewing of example notebooks? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3630/reactions"", ""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 499481465,MDU6SXNzdWU0OTk0ODE0NjU=,3351,"custom ""coordinates"" attribute",2448579,closed,0,,,0,2019-09-27T14:32:58Z,2019-12-10T16:02:01Z,2019-12-10T16:02:01Z,MEMBER,,,,"This set of lines prevents users from writing files with a handcrafted `coordinates` attribute. This is useful when not all non-dimensional coordinates in a Dataset are appropriate for every data variable in a dataset. https://github.com/pydata/xarray/blob/16fdac93c09036b44f90201d9078b17808f819e0/xarray/conventions.py#L667-L675 It seems to me like we should only automatically set ""coordinates"" when `""coordinates"" not in attrs`. We could raise a warning saying that setting `attrs[""coordinates""]` prevents full roundtripping of Datasets. What do you think? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3351/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 533578755,MDU6SXNzdWU1MzM1Nzg3NTU=,3599,map_blocks graph construction bug,2448579,closed,0,,,0,2019-12-05T20:23:06Z,2019-12-07T04:30:19Z,2019-12-07T04:30:19Z,MEMBER,,,,"Just making a new issue for #3598 --------------- The tests for https://github.com/pydata/xarray/pull/3584 fail on `dask == 2.8.1` with this interesting bug. Here's a reproducible example. ``` python import dask import xarray as xr ds = xr.Dataset({'x': (('y',), dask.array.ones(10, chunks=(3,)))}) mapped = ds.map_blocks(lambda x: x) mapped.compute() # works xr.testing.assert_equal(mapped, ds) # does not work xr.testing.assert_equal(mapped, ds.compute()) # works xr.testing.assert_equal(mapped.compute(), ds) # works xr.testing.assert_equal(mapped.compute(), ds.compute()) # works ``` The traceback is ``` ~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/array/optimization.py in optimize(dsk, keys, fuse_keys, fast_functions, inline_functions_fast_functions, rename_fused_keys, **kwargs) 41 if isinstance(dsk, HighLevelGraph): 42 dsk = optimize_blockwise(dsk, keys=keys) ---> 43 dsk = fuse_roots(dsk, keys=keys) 44 45 # Low level task optimizations ~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/blockwise.py in fuse_roots(graph, keys) 819 isinstance(layer, Blockwise) 820 and len(deps) > 1 --> 821 and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1 822 and all(len(dependents[dep]) == 1 for dep in deps) 823 ): ~/miniconda3/envs/dcpy/lib/python3.7/site-packages/dask/blockwise.py in (.0) 819 isinstance(layer, Blockwise) 820 and len(deps) > 1 --> 821 and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1 822 and all(len(dependents[dep]) == 1 for dep in deps) 823 ): KeyError: 'lambda-6720ab0e3639d5c63fc06dfc66a3ce47-x' ``` This key is not in `dependencies`. From https://github.com/dask/dask/blob/67fb5363009c583c175cb577776a4f2f4e811410/dask/blockwise.py#L816-L826 ``` python for name, layer in graph.layers.items(): deps = graph.dependencies[name] if ( isinstance(layer, Blockwise) and len(deps) > 1 and not any(dependencies[dep] for dep in deps) # no need to fuse if 0 or 1 and all(len(dependents[dep]) == 1 for dep in deps) ): new = toolz.merge(layer, *[layers[dep] for dep in deps]) new, _ = fuse(new, keys, ave_width=len(deps)) ``` I'm not sure whether this is a bug in `fuse_roots`, `HighLevelGraph.from_collections` or in how `map_blocks` calls `HighLevelGraph.from_collections` here: https://github.com/pydata/xarray/blob/69c85b85a1ef5d13b6ca51b6b655a1f719cc5abf/xarray/core/parallel.py#L315","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3599/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 494095795,MDU6SXNzdWU0OTQwOTU3OTU=,3311,optimize compatibility checks in merge.unique_variable,2448579,closed,0,,,0,2019-09-16T14:43:39Z,2019-11-05T15:41:15Z,2019-11-05T15:41:15Z,MEMBER,,,,"Currently `merge.unique_variable` does this ``` if equals is None: out = out.compute() for var in variables[1:]: equals = getattr(out, compat)(var) if not equals: break ``` `out` (`=variables[0]`) is always computed though it may not be necessary. One solution would be to loop through once checking attrs, shapes and `_data`. If these checks were satisfied, then we execute the above code","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3311/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 491898181,MDU6SXNzdWU0OTE4OTgxODE=,3300,test for h5netcdf invalid_netcdf warning is failing,2448579,closed,0,,,0,2019-09-10T21:05:48Z,2019-09-13T15:39:41Z,2019-09-13T15:39:41Z,MEMBER,,,,"This passes locally with `h5py==2.9.0` but the CI is running `h5py==2.10.0`. It's probably issuing an extra warning. ``` =================================== FAILURES =================================== _____________ TestH5NetCDFData.test_complex[None-FutureWarning-1] ______________ self = invalid_netcdf = None, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError _____________ TestH5NetCDFData.test_complex[False-FutureWarning-1] _____________ self = invalid_netcdf = False, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError __________________ TestH5NetCDFData.test_complex[True-None-0] __________________ self = invalid_netcdf = True, warns = None, num_warns = 0 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 1 == 0 E + where 1 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError __________ TestH5NetCDFFileObject.test_complex[None-FutureWarning-1] ___________ self = invalid_netcdf = None, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError __________ TestH5NetCDFFileObject.test_complex[False-FutureWarning-1] __________ self = invalid_netcdf = False, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError _______________ TestH5NetCDFFileObject.test_complex[True-None-0] _______________ self = invalid_netcdf = True, warns = None, num_warns = 0 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 1 == 0 E + where 1 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError __________ TestH5NetCDFViaDaskData.test_complex[None-FutureWarning-1] __________ self = invalid_netcdf = None, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError _________ TestH5NetCDFViaDaskData.test_complex[False-FutureWarning-1] __________ self = invalid_netcdf = False, warns = , num_warns = 1 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 2 == 1 E + where 2 = len(WarningsChecker(record=True)) xarray/tests/test_backends.py:2185: AssertionError ______________ TestH5NetCDFViaDaskData.test_complex[True-None-0] _______________ self = invalid_netcdf = True, warns = None, num_warns = 0 @pytest.mark.filterwarnings(""ignore:complex dtypes are supported by h5py"") @pytest.mark.parametrize( ""invalid_netcdf, warns, num_warns"", [(None, FutureWarning, 1), (False, FutureWarning, 1), (True, None, 0)], ) def test_complex(self, invalid_netcdf, warns, num_warns): expected = Dataset({""x"": (""y"", np.ones(5) + 1j * np.ones(5))}) save_kwargs = {""invalid_netcdf"": invalid_netcdf} with pytest.warns(warns) as record: with self.roundtrip(expected, save_kwargs=save_kwargs) as actual: assert_equal(expected, actual) > assert len(record) == num_warns E assert 1 == 0 E + where 1 = len(WarningsChecker(record=True)) ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3300/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 435787982,MDU6SXNzdWU0MzU3ODc5ODI=,2913,Document xarray data model,2448579,open,0,,,0,2019-04-22T16:23:41Z,2019-04-22T16:23:41Z,,MEMBER,,,,It would be nice to have a separate page that detailed this for users unfamiliar with netCDF. ,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2913/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 391440480,MDU6SXNzdWUzOTE0NDA0ODA=,2610,docs build errors,2448579,closed,0,,,0,2018-12-16T06:45:58Z,2018-12-17T21:57:36Z,2018-12-17T21:57:36Z,MEMBER,,,,"We're seeing a lot of errors in the docs build on Travis though the build is reported successful?! https://travis-ci.org/pydata/xarray/jobs/468535214","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2610/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue