id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 2052840951,I_kwDOAMm_X856W933,8566,Use `ddof=1` for `std` & `var`,5635139,open,0,,,2,2023-12-21T17:47:21Z,2023-12-27T16:58:46Z,,MEMBER,,,,"### What is your issue? I've discussed this a bunch with @dcherian (though I'm not sure he necessarily agrees, I'll let him comment) Currently xarray uses `ddof=0` for `std` & `var`. This is: - Rarely what someone actually wants — xarray data is almost always a sample of some underlying distribution, for which `ddof=1` is correct - Inconsistent with [pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.var.html) OTOH: - It is consistent with [numpy](https://numpy.org/doc/stable/reference/generated/numpy.var.html) - It wouldn't be a painless change — folks who don't read deprecation messages would see values change very slightly Any thoughts?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8566/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 907845790,MDU6SXNzdWU5MDc4NDU3OTA=,5413,Does the PyPI release job fire twice for each release?,5635139,closed,0,,,2,2021-06-01T04:01:17Z,2023-12-04T19:22:32Z,2023-12-04T19:22:32Z,MEMBER,,,," I was attempting to copy the great work here for numbagg and spotted this! Do we fire twice for each release? Maybe that's fine though? https://github.com/pydata/xarray/actions/workflows/pypi-release.yaml ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5413/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1203835220,I_kwDOAMm_X85HwRFU,6484,Should we raise a more informative error on no zarr dir?,5635139,closed,0,,,2,2022-04-13T22:05:07Z,2022-09-20T22:38:46Z,2022-09-20T22:38:46Z,MEMBER,,,,"### What happened? Currently if someone supplies a path that doesn't exist, we get quite a long stack trace, without really saying that the path doesn't exist. ### What did you expect to happen? Possibly a `FileNotFoundError` ### Minimal Complete Verifiable Example ```Python xr.open_zarr('x.zarr') ``` ### Relevant log output ```Python In [1]: xr.open_zarr('x.zarr') :1: RuntimeWarning: Failed to open Zarr store with consolidated metadata, falling back to try reading non-consolidated metadata. This is typically much slower for opening a dataset. To silence this warning, consider: 1. Consolidating metadata in this existing store with zarr.consolidate_metadata(). 2. Explicitly setting consolidated=False, to avoid trying to read consolidate metadata, or 3. Explicitly setting consolidated=True, to raise an error in this case instead of falling back to try reading non-consolidated metadata. xr.open_zarr('x.zarr') --------------------------------------------------------------------------- KeyError Traceback (most recent call last) ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/xarray/backends/zarr.py in open_group(cls, store, mode, synchronizer, group, consolidated, consolidate_on_close, chunk_store, storage_options, append_dim, write_region, safe_chunks, stacklevel) 347 try: --> 348 zarr_group = zarr.open_consolidated(store, **open_kwargs) 349 except KeyError: ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/zarr/convenience.py in open_consolidated(store, metadata_key, mode, **kwargs) 1186 # setup metadata store -> 1187 meta_store = ConsolidatedMetadataStore(store, metadata_key=metadata_key) 1188 ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/zarr/storage.py in __init__(self, store, metadata_key) 2643 # retrieve consolidated metadata -> 2644 meta = json_loads(store[metadata_key]) 2645 ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/zarr/storage.py in __getitem__(self, key) 894 else: --> 895 raise KeyError(key) 896 KeyError: '.zmetadata' During handling of the above exception, another exception occurred: GroupNotFoundError Traceback (most recent call last) in () ----> 1 xr.open_zarr('x.zarr') ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/xarray/backends/zarr.py in open_zarr(store, group, synchronizer, chunks, decode_cf, mask_and_scale, decode_times, concat_characters, decode_coords, drop_variables, consolidated, overwrite_encoded_chunks, chunk_store, storage_options, decode_timedelta, use_cftime, **kwargs) 750 } 751 --> 752 ds = open_dataset( 753 filename_or_obj=store, 754 group=group, ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, backend_kwargs, *args, **kwargs) 493 494 overwrite_encoded_chunks = kwargs.pop(""overwrite_encoded_chunks"", None) --> 495 backend_ds = backend.open_dataset( 496 filename_or_obj, 497 drop_variables=drop_variables, ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/xarray/backends/zarr.py in open_dataset(self, filename_or_obj, mask_and_scale, decode_times, concat_characters, decode_coords, drop_variables, use_cftime, decode_timedelta, group, mode, synchronizer, consolidated, chunk_store, storage_options, stacklevel) 798 799 filename_or_obj = _normalize_path(filename_or_obj) --> 800 store = ZarrStore.open_group( 801 filename_or_obj, 802 group=group, ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/xarray/backends/zarr.py in open_group(cls, store, mode, synchronizer, group, consolidated, consolidate_on_close, chunk_store, storage_options, append_dim, write_region, safe_chunks, stacklevel) 363 stacklevel=stacklevel, 364 ) --> 365 zarr_group = zarr.open_group(store, **open_kwargs) 366 elif consolidated: 367 # TODO: an option to pass the metadata_key keyword ~/Library/Caches/pypoetry/virtualenvs/-x204KUJE-py3.9/lib/python3.9/site-packages/zarr/hierarchy.py in open_group(store, mode, cache_attrs, synchronizer, path, chunk_store, storage_options) 1180 if contains_array(store, path=path): 1181 raise ContainsArrayError(path) -> 1182 raise GroupNotFoundError(path) 1183 1184 elif mode == 'w': GroupNotFoundError: group not found at path '' ``` ### Anything else we need to know? _No response_ ### Environment INSTALLED VERSIONS ------------------ commit: None python: 3.9.12 (main, Mar 26 2022, 15:44:31) [Clang 13.1.6 (clang-1316.0.21.2)] python-bits: 64 OS: Darwin OS-release: 21.3.0 machine: arm64 processor: arm byteorder: little LC_ALL: en_US.UTF-8 LANG: None LOCALE: ('en_US', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.1 numpy: 1.22.3 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.1 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.12.0 distributed: 2021.12.0 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2021.11.1 cupy: None pint: None sparse: None setuptools: 60.9.3 pip: 21.3.1 conda: None pytest: 6.2.5 IPython: 7.32.0 sphinx: None","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6484/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1200356907,I_kwDOAMm_X85Hi_4r,6473,RTD concurrency limit,5635139,closed,0,,,2,2022-04-11T18:14:05Z,2022-04-19T06:29:24Z,2022-04-19T06:29:24Z,MEMBER,,,,"### What is your issue? From https://github.com/pydata/xarray/pull/6472, and some PRs this weekend: Is anyone familiar with what's going on with RTD? Did our concurrency limit drop? Are there alternatives (e.g. running the tests on GHA even if the actual docs get built on RTD?). If we have to pay RTD for a subscription for a bit until we make changes then we could do that (I'm happy to given my recently poor contribution track-record!)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6473/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1207211171,I_kwDOAMm_X85H9JSj,6499,Added `automerge`,5635139,closed,0,,,2,2022-04-18T16:24:35Z,2022-04-18T18:21:39Z,2022-04-18T16:24:41Z,MEMBER,,,,"### What is your issue? @pydata/xarray Because our pipeline takes a while, it can be helpful to have an option to ""merge when tests pass"" — I've now set that up. So you can click here and it'll do just that. Someone annoyingly / confusingly, the ""required checks"" need to be specified manually, in https://github.com/pydata/xarray/settings/branch_protection_rules/2465574 — there's no option for just ""all checks"". So if we change the checks — e.g. add Python 3.11 — that list needs to be updated. If we remove a check from the our CI and don't update the list, it won't be possible to merge the PR without clicking the red ""Admin Override"" box — so we should keep it up to date. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6499/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 895918276,MDU6SXNzdWU4OTU5MTgyNzY=,5348,v0.18.2,5635139,closed,0,,,2,2021-05-19T21:21:18Z,2021-05-20T01:51:12Z,2021-05-19T21:35:47Z,MEMBER,,,,"I'm about to release this as v0.18.2: https://github.com/pydata/xarray/compare/v0.18.1...max-sixty:release-0.18.2?expand=1 given https://github.com/pydata/xarray/issues/5346 Let me know any thoughts @pydata/xarray , thanks","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5348/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 728893769,MDU6SXNzdWU3Mjg4OTM3Njk=,4535,Support operations with pandas Offset objects,5635139,closed,0,,,2,2020-10-24T22:49:57Z,2021-03-06T23:02:01Z,2021-03-06T23:02:01Z,MEMBER,,,," **Is your feature request related to a problem? Please describe.** Currently xarray objects containting datetimes don't operate with pandas' offset objects: ```python times = pd.date_range(""2000-01-01"", freq=""6H"", periods=10) ds = xr.Dataset( { ""foo"": ([""time"", ""x"", ""y""], np.random.randn(10, 5, 3)), ""bar"": (""time"", np.random.randn(10), {""meta"": ""data""}), ""time"": times, } ) ds.attrs[""dsmeta""] = ""dsdata"" ds.resample(time=""24H"").mean(""time"").time + to_offset(""8H"") ``` raises: ``` --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in ----> 1 ds.resample(time=""24H"").mean(""time"").time + to_offset(""8H"") /usr/local/lib/python3.8/site-packages/xarray/core/dataarray.py in func(self, other) 2763 2764 variable = ( -> 2765 f(self.variable, other_variable) 2766 if not reflexive 2767 else f(other_variable, self.variable) /usr/local/lib/python3.8/site-packages/xarray/core/variable.py in func(self, other) 2128 with np.errstate(all=""ignore""): 2129 new_data = ( -> 2130 f(self_data, other_data) 2131 if not reflexive 2132 else f(other_data, self_data) TypeError: unsupported operand type(s) for +: 'numpy.ndarray' and 'pandas._libs.tslibs.offsets.Hour' ``` This is an issue because pandas resampling has deprecated `loffset` — from our test suite: ``` xarray/tests/test_dataset.py::TestDataset::test_resample_loffset /Users/maximilian/workspace/xarray/xarray/tests/test_dataset.py:3844: FutureWarning: 'loffset' in .resample() and in Grouper() is deprecated. >>> df.resample(freq=""3s"", loffset=""8H"") becomes: >>> from pandas.tseries.frequencies import to_offset >>> df = df.resample(freq=""3s"").mean() >>> df.index = df.index.to_timestamp() + to_offset(""8H"") ds.bar.to_series().resample(""24H"", loffset=""-12H"").mean() ``` ...and so we'll need to support something like this in order to maintain existing behavior. **Describe the solution you'd like** I'm not completely sure; I think probably supporting the operations between xarray objects containing datetime objects and pandas' offset objects. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4535/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 572995385,MDU6SXNzdWU1NzI5OTUzODU=,3811,Don't warn on empty reductions,5635139,closed,0,,,2,2020-02-28T20:45:38Z,2021-02-21T23:05:46Z,2021-02-21T23:05:46Z,MEMBER,,,," Numpy warns when computing over an all-NaN slice. We handle that case reasonably and so should handle and discard the warning. #### MCVE Code Sample ```python In [1]: import xarray as xr In [2]: import numpy as np In [3]: da = xr.DataArray(np.asarray([np.nan]*3)) In [4]: da Out[4]: array([nan, nan, nan]) Dimensions without coordinates: dim_0 In [6]: da.mean() [...]/python3.6/site-packages/xarray/core/nanops.py:142: RuntimeWarning: Mean of empty slice return np.nanmean(a, axis=axis, dtype=dtype) Out[6]: array(nan) ``` #### Expected Output No warning #### Problem Description Somewhat discussed in https://github.com/pydata/xarray/issues/1164, and https://github.com/pydata/xarray/issues/1652, but starting a separate issue as it's more important than just noise in the test suite, and not covered by the existing work on comparisons & arithmetic #### Output of ``xr.show_versions()``
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3811/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 576692586,MDU6SXNzdWU1NzY2OTI1ODY=,3837,Should we run tests on docstrings?,5635139,closed,0,,,2,2020-03-06T04:35:16Z,2020-09-11T12:34:34Z,2020-09-11T12:34:34Z,MEMBER,,,,"Currently almost none of the docstrings pass running `pytest --doctest-modules xarray/core`, though mostly for easy reasons. Should we run these in CI? I've recently started using docstring tests in another project, and they've work pretty well. CC @keewis ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3837/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 132579684,MDU6SXNzdWUxMzI1Nzk2ODQ=,755,count docstring mistakenly includes skipna,5635139,closed,0,,,2,2016-02-10T00:49:34Z,2020-07-24T16:09:25Z,2020-07-24T16:09:25Z,MEMBER,,,,"Is this a mistake or am I missing something? http://xray.readthedocs.org/en/stable/generated/xarray.DataArray.count.html?highlight=count#xarray.DataArray.count > skipna : bool, optional > If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64). ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/755/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 586450690,MDU6SXNzdWU1ODY0NTA2OTA=,3881,Flaky test: test_uamiv_format_write,5635139,closed,0,,,2,2020-03-23T19:13:34Z,2020-03-23T20:32:15Z,2020-03-23T20:32:15Z,MEMBER,,,,"I've seen a couple of failures recently on this test. Flaky tests are really annoying and would be great to fix or if impossible, remove it. Does anyone have any ideas what's causing this? ``` ________________ TestPseudoNetCDFFormat.test_uamiv_format_write ________________ self = def test_uamiv_format_write(self): fmtkw = {""format"": ""uamiv""} expected = open_example_dataset( ""example.uamiv"", engine=""pseudonetcdf"", backend_kwargs=fmtkw ) with self.roundtrip( expected, save_kwargs=fmtkw, open_kwargs={""backend_kwargs"": fmtkw}, allow_cleanup_failure=True, ) as actual: > assert_identical(expected, actual) E AssertionError: Left and right Dataset objects are not identical E E E E Differing attributes: E L WTIME: 190117 E R WTIME: 190118 xarray/tests/test_backends.py:3563: AssertionError ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3881/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 508791645,MDU6SXNzdWU1MDg3OTE2NDU=,3414,Allow ellipsis in place of xr.ALL_DIMS?,5635139,closed,0,,,2,2019-10-18T00:44:48Z,2019-10-28T21:14:42Z,2019-10-28T21:14:42Z,MEMBER,,,,"@crusaderky had a [good idea](https://github.com/pydata/xarray/issues/1081#issuecomment-540477534) to allow ellipsis (`...`) as a placeholder for 'other dims' in transpose. What about using it as a placeholder for `xr.ALL_DIMS` in groupby etc operations? I find it nicer than custom sentinel values, and I think should be fairly low-confusion—thoughts?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3414/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 467015096,MDU6SXNzdWU0NjcwMTUwOTY=,3098,Codecov bot comments?,5635139,closed,0,,,2,2019-07-11T17:21:46Z,2019-07-18T01:12:38Z,2019-07-18T01:12:38Z,MEMBER,,,,"ref https://github.com/pydata/xarray/pull/3090#issuecomment-510323490 Do we want the bot commenting on the PR, at least while the early checks are wrong? People can always click on `Details` in the Codecov check (e.g. https://codecov.io/gh/pydata/xarray/compare/8f0d9e5c9909c93a90306ed7cb5a80c1c2e1c97d...ab6960f623017afdc99c34bcbb69b402aea3f7d4/diff) to see a full report. Happy to PR to disable, lmk","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3098/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 423511704,MDU6SXNzdWU0MjM1MTE3MDQ=,2833,Integrate has undefined name 'dim',5635139,closed,0,,,2,2019-03-20T23:09:19Z,2019-07-05T07:10:37Z,2019-07-05T07:10:37Z,MEMBER,,,,"https://github.com/pydata/xarray/blob/master/xarray/core/dataset.py#L4085 Should that be called `coord` or `dim`? Currently there's a variable that's undefined: ```python raise ValueError('Coordinate {} does not exist.'.format(dim)) ``` I would have made a quick fix but not sure the correct name","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2833/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 448340294,MDU6SXNzdWU0NDgzNDAyOTQ=,2990,Some minor errors in repo / flake8,5635139,closed,0,,,2,2019-05-24T20:24:04Z,2019-06-24T18:18:25Z,2019-06-24T18:18:24Z,MEMBER,,,,"Currently we use pycodestyle: https://github.com/pydata/xarray/blob/ccd0b047ea8ca89c68ab6cfa942557e676e7d402/.travis.yml#L63 I think we used to use flake8. I can't find / remember the reason we moved to pycodestyle. master has some non-trivial issues that flake would catch, including a test overwritting another and undefined variables: ``` flake8 xarray --ignore=I,W503,W504,F401,E265,E402 xarray/core/options.py:62:8: F632 use ==/!= to compare str, bytes, and int literals xarray/core/dataset.py:4148:69: F821 undefined name 'dim' xarray/backends/netCDF4_.py:177:12: F632 use ==/!= to compare str, bytes, and int literals xarray/tests/test_dataarray.py:1264:9: F841 local variable 'foo' is assigned to but never used xarray/tests/test_dataarray.py:1270:18: F821 undefined name 'x' xarray/tests/test_dataarray.py:1301:5: F811 redefinition of unused 'test_reindex_fill_value' from line 1262 xarray/tests/test_dataarray.py:1647:16: F632 use ==/!= to compare str, bytes, and int literals xarray/tests/test_dataarray.py:1648:16: F632 use ==/!= to compare str, bytes, and int literals xarray/tests/test_dataset.py:4759:8: F632 use ==/!= to compare str, bytes, and int literals xarray/tests/test_dataset.py:4761:10: F632 use ==/!= to compare str, bytes, and int literals xarray/tests/test_distributed.py:62:9: F811 redefinition of unused 'loop' from line 12 xarray/tests/test_distributed.py:92:9: F811 redefinition of unused 'loop' from line 12 xarray/tests/test_distributed.py:117:49: F811 redefinition of unused 'loop' from line 12 xarray/tests/test_distributed.py:141:53: F811 redefinition of unused 'loop' from line 12 xarray/tests/test_distributed.py:152:51: F811 redefinition of unused 'loop' from line 12 ``` Happy to fix these in a PR. For ensuring these don't crop up again, any objection to flake8?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2990/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 365526259,MDU6SXNzdWUzNjU1MjYyNTk=,2451,"Shift changes non-float arrays to object, even for shift=0",5635139,closed,0,,,2,2018-10-01T15:50:38Z,2019-03-04T16:31:57Z,2019-03-04T16:31:57Z,MEMBER,,,," ```python In [15]: xr.DataArray(np.random.randint(2,size=(100,100)).astype(bool)).shift(dim_0=0) Out[15]: array([[False, True, True, ..., True, True, False], [False, True, False, ..., False, True, True], [False, True, False, ..., False, True, False], ..., [False, True, False, ..., False, True, True], [True, False, True, ..., False, False, False], [False, True, True, ..., True, True, False]], dtype=object) # <-- could be bool Dimensions without coordinates: dim_0, dim_1 ``` #### Problem description This causes memory bloat #### Expected Output As above with `dtype=bool` #### Output of ``xr.show_versions()``
In [16]: xr.show_versions() INSTALLED VERSIONS ------------------ commit: f9c4169150286fa1aac020ab965380ed21fe1148 python: 2.7.15.final.0 python-bits: 64 OS: Darwin OS-release: 18.0.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None xarray: 0.10.9+12.gf9c41691 pandas: 0.22.0 numpy: 1.14.2 scipy: 1.0.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.1.2 cartopy: None seaborn: 0.8.1 setuptools: 39.2.0 pip: 18.0 conda: None pytest: 3.6.3 IPython: 5.8.0 sphinx: None
The `shift=0` is mainly theoretical. To avoid casting to `object` in practical scenarios, we could add a `fill_value` argument (e.g. `fill_value=False`) and fill with that rather than `NaN` CC @Ivocrnkovic","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2451/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 367424540,MDU6SXNzdWUzNjc0MjQ1NDA=,2468,LooseVersion check on xarray in tests seems unstable,5635139,closed,0,,,2,2018-10-06T05:07:47Z,2018-10-10T13:47:23Z,2018-10-10T13:47:23Z,MEMBER,,,,"There's an elegant check against the xarray version to decide whether to run a test, so the test 'comes online' at 0.12: https://github.com/pydata/xarray/blob/638b251c622359b665208276a2cb23b0fbc5141b/xarray/tests/test_dataarray.py#L2029 But unfortunately, this seems very unstable in tests, because without a release, `LooseVersion` can't interpret the strings correctly (e.g. `LooseVersion ('0.10.9+29.g33d9391a')`) - A lot of the time it raises: https://travis-ci.org/max-sixty/xarray/jobs/437913418#L1036 - Occasionally it runs the test, failing to return that we're prior to 0.12: https://travis-ci.org/max-sixty/xarray/jobs/437914645#L5036 Here's the bug in the python issue tracker: https://bugs.python.org/issue14894 Is that synopsis correct? Should we attempt to take another approach? I'll disable it in my current check so tests can pass, but lmk thoughts.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2468/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 271131362,MDU6SXNzdWUyNzExMzEzNjI=,1691,Coordinates passed as sets raise with a bad error message,5635139,closed,0,,,2,2017-11-03T22:06:28Z,2018-08-08T15:56:57Z,2018-08-08T15:56:57Z,MEMBER,,,,"If a coordinate is passed as a `set`, xr raises with a bad error message: ```python In [12]: xr.Dataset(dict(date=[1,2,3], sec={4})) --------------------------------------------------------------------------- MissingDimensionsError Traceback (most recent call last) in () ----> 1 xr.Dataset(dict(date=[1,2,3], sec={4})) /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/dataset.py in __init__(self, data_vars, coords, attrs, compat) 360 coords = {} 361 if data_vars is not None or coords is not None: --> 362 self._set_init_vars_and_dims(data_vars, coords, compat) 363 if attrs is not None: 364 self.attrs = attrs /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/dataset.py in _set_init_vars_and_dims(self, data_vars, coords, compat) 378 379 variables, coord_names, dims = merge_data_and_coords( --> 380 data_vars, coords, compat=compat) 381 382 self._variables = variables /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/merge.py in merge_data_and_coords(data, coords, compat, join) 363 objs = [data, coords] 364 explicit_coords = coords.keys() --> 365 return merge_core(objs, compat, join, explicit_coords=explicit_coords) 366 367 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/merge.py in merge_core(objs, compat, join, priority_arg, explicit_coords, indexes) 425 coerced = coerce_pandas_values(objs) 426 aligned = deep_align(coerced, join=join, copy=False, indexes=indexes) --> 427 expanded = expand_variable_dicts(aligned) 428 429 coord_names, noncoord_names = determine_coords(coerced) /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/merge.py in expand_variable_dicts(list_of_variable_dicts) 211 var_dicts.append(coords) 212 --> 213 var = as_variable(var, name=name) 214 sanitized_vars[name] = var 215 /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xarray/core/variable.py in as_variable(obj, name) 103 'cannot set variable %r with %r-dimensional data ' 104 'without explicit dimension names. Pass a tuple of ' --> 105 '(dims, data) instead.' % (name, data.ndim)) 106 obj = Variable(name, obj, fastpath=True) 107 else: MissingDimensionsError: cannot set variable 'sec' with 0-dimensional data without explicit dimension names. Pass a tuple of (dims, data) instead. ``` But OK if a list: ```python In [13]: xr.Dataset(dict(date=[1,2,3], sec=[4])) Out[13]: Dimensions: (date: 3, sec: 1) Coordinates: * date (date) int64 1 2 3 * sec (sec) int64 4 Data variables: *empty* ``` #### Problem description There may be reasons to not allow sets: they're not ordered, so unless you're constructing your data using the `coords`, you'll get random results The error message should be better though. And I would vote to handle sets the same as lists #### Expected Output #### Output of ``xr.show_versions()``
INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Darwin OS-release: 17.0.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.0rc1-2-gf83361c pandas: 0.21.0 numpy: 1.13.3 scipy: 0.19.1 netCDF4: None h5netcdf: None Nio: None bottleneck: 1.2.1 cyordereddict: None dask: None matplotlib: 2.0.2 cartopy: None seaborn: 0.8.1 setuptools: 36.5.0 pip: 9.0.1 conda: None pytest: 3.2.3 IPython: 6.2.1 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1691/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 161991202,MDU6SXNzdWUxNjE5OTEyMDI=,890,BUG: Dataset constructor puts lists in coords rather that data_vars,5635139,closed,0,,,2,2016-06-23T18:28:44Z,2018-07-31T18:28:29Z,2018-07-31T18:28:29Z,MEMBER,,,,"I'd expect `a` to be a data_vars rather than a coord here: ``` python In [9]: xr.Dataset(data_vars={'a': [2,3]}, attrs={'name':'hello'}) Out[9]: Dimensions: (a: 2) Coordinates: * a (a) int64 2 3 Data variables: *empty* Attributes: name: hello ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/890/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 211860631,MDU6SXNzdWUyMTE4NjA2MzE=,1294,python 3.6 tests break with bottleneck installed,5635139,closed,0,,,2,2017-03-04T06:35:24Z,2017-12-10T01:52:34Z,2017-12-10T01:52:34Z,MEMBER,,,,"Installing 3.6 environment (from the ci path in xarray): tests pass on master Then installing bottleneck: 3 tests in test_dataarray.py fail on master I can debug further unless anyone has a view","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1294/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 125092434,MDU6SXNzdWUxMjUwOTI0MzQ=,708,'to_array' creates a read-only numpy array,5635139,closed,0,,,2,2016-01-06T01:46:12Z,2016-01-06T02:42:32Z,2016-01-06T02:42:32Z,MEMBER,,,,"Is this intended? It's creating some problems downstream with pandas, but maybe that's a pandas issue? Note the `WRITEABLE : False` here: ``` python In [126]: ds=xray.Dataset({'a':xray.DataArray(pd.np.random.rand(5,3))}, coords={'b': xray.DataArray(pd.np.random.rand(5))}) In [127]: ds.to_array('d').b.values.flags Out[127]: C_CONTIGUOUS : True F_CONTIGUOUS : True OWNDATA : False WRITEABLE : False ALIGNED : True UPDATEIFCOPY : False ``` Without the `to_array`, it's fine: ``` python In [128]: ds.b.values.flags Out[128]: C_CONTIGUOUS : True F_CONTIGUOUS : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False ``` xref https://github.com/pydata/pandas/issues/11502 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/708/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 124176573,MDU6SXNzdWUxMjQxNzY1NzM=,689,Should Dataset enforce some ordering of dims in its variables?,5635139,closed,0,,,2,2015-12-29T07:41:02Z,2015-12-29T21:20:56Z,2015-12-29T21:20:56Z,MEMBER,,,,"I'm not sure on this one. I'm currently having a bunch of issues with this sort of Dataset: (notice the dims are `(d, c)` and `(c, d)` for different variables) ``` python Dimensions: (c: 193, d: 6781) Coordinates: * d (d) object 5218 5219 5220 5221 5222 5223 5224 ... * c (c) object LDS. ... j (c, d) bool False False False False ... Data variables: r (d, c) float64 nan -0.05083 nan ... s (d, c) float64 nan -0.05083 nan ... n (c, d) float64 nan nan nan nan nan ... ``` In my case, this is particularly painful when passing the result of `ds.r.to_pandas()` into a function expecting a DataFrame with a certain orientation when that orientation isn't reliable. Is this a problem generally? If it is, I could imagine a few solutions - enforce ordering, offer a method on a DataSet to align the dims, offer a `kwarg` on `.to_pandas()` to allow specifying the dims-axis mapping, etc ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/689/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 122171841,MDU6SXNzdWUxMjIxNzE4NDE=,679,Is 'name' an xray supported attribute? ,5635139,closed,0,,,2,2015-12-15T01:34:48Z,2015-12-15T03:16:37Z,2015-12-15T03:16:37Z,MEMBER,,,,"If it is, a Dataset constructor should take a list of DataArrays, and use their names as keys? (and anywhere else you need to provide a dict-like mapping with names) If it's not, we potentially shouldn't be using it in the internals. I think it's the first, given it's in the [docs](http://xray.readthedocs.org/en/stable/generated/xray.DataArray.name.html) (although not [throughout](http://xray.readthedocs.org/en/stable/data-structures.html#dataarray) the docs). ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/679/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 115979105,MDU6SXNzdWUxMTU5NzkxMDU=,652,ENH: Apply numpy function to named axes,5635139,closed,0,,,2,2015-11-09T22:11:19Z,2015-11-10T16:18:24Z,2015-11-10T16:18:24Z,MEMBER,,,,"I'm currently transitioning sequences of pandas `Panel`s over to xray `Dataset`s. Part of our process applies a set of functions to Panels; for example: ``` python panel = panel.apply(lambda x: x.rank(ascending=False), axis=(1, 2)) df = np.nanpercentile(panel, q=75, axis=2) ``` One of the benefits of xray is the clarity that comes from named axes. Is there a way of applying a function over named axes? For example: ``` python result = data_array.apply(np.argsort, axis=('Risk', 'Dates')) result = data_array.apply(np.nanpercentile, axis='Dates') ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/652/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue