id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 2241228882,PR_kwDOAMm_X85sjOh9,8936,MAINT: use sphinxext-rediraffe conda install,17162724,closed,0,,,1,2024-04-13T02:11:07Z,2024-04-13T02:53:53Z,2024-04-13T02:53:48Z,CONTRIBUTOR,,0,pydata/xarray/pulls/8936," - [ ] Closes #xxxx - [ ] Tests added - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8936/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 2176449106,PR_kwDOAMm_X85pGrBh,8816,DOC: link to zarr.convenience.consolidate_metadata,17162724,closed,0,,,2,2024-03-08T17:14:19Z,2024-03-08T19:20:03Z,2024-03-08T18:47:14Z,CONTRIBUTOR,,0,pydata/xarray/pulls/8816,"I wanted to know what `zarr.convenience.consolidate_metadata` does which motivated me to add a link to it in the docstring to make it easier for others (and future me) to discover it. I sorted `intersphinx_mapping` to make it easier to docs from other projects. I also added :func:`zarr.convenience.consolidate_metadata` in the doc string. Testing if it actually works is WIP","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1319964729,I_kwDOAMm_X85OrRA5,6840,better warning message for xarray/core/dataset.py:2060: SerializationWarning,17162724,open,0,,,0,2022-07-27T18:26:55Z,2024-01-25T18:02:57Z,,CONTRIBUTOR,,,,"### Is your feature request related to a problem? I was trying to store a zarr fstore and I got: ``` /opt/userenvs/ray.bell/autofront/lib/python3.9/site-packages/xarray/core/dataset.py:2060: SerializationWarning: saving variable None with floating point data as an integer dtype without any _FillValue to use for NaNs return to_zarr( # type: ignore ``` ~~What I was seeing was a time coord which had encoding 'dtype': dtype('int64'). Storing the time this way lost the HH:MM on my datetime array.~~ I think I am seeing 'units': 'days since 2022-MM-DD 00:00:00' whereas it should be `hours` for my data. ### Describe the solution you'd like The message captures the variable/coord name instead of **None** It could offer a solution such as ""to store floating point data you could possibly use `encoding={variable/coord: {""dtype"": float, ""_Fillvalue"": np.nan}`"" ### Describe alternatives you've considered _No response_ ### Additional context I'll try to come back and write a MCVE WIP: you can get the warning message by doing: ``` ds = xr.tutorial.open_dataset(""air_temperature"") ds[""time""].encoding # {'units': 'hours since 1800-01-01', 'calendar': 'standard', 'dtype': dtype('float32')} ds.isel(time=0).to_zarr(""tmp0.zarr"", mode=""w"") ``` /Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/dataset.py:2060: SerializationWarning: saving variable None with floating point data as an integer dtype without any _FillValue to use for NaNs return to_zarr( # type: ignore My guess if one of lat, lon, time, air is raising that message Can't find the associated code in the repo that raises this. 2060 is to_zarr AFICT (https://github.com/pydata/xarray/blob/main/xarray/core/dataset.py#L2066)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6840/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 815825360,MDU6SXNzdWU4MTU4MjUzNjA=,4952,Feature request: dropna() along multiple dimensions,17162724,open,0,,,3,2021-02-24T20:30:39Z,2023-08-16T18:09:32Z,,CONTRIBUTOR,,,," **Is your feature request related to a problem? Please describe.** In the docs (http://xarray.pydata.org/en/stable/generated/xarray.Dataset.dropna.html) it says Dropping along multiple dimensions simultaneously is not yet supported. But I couldn't find a GitHub Issue open it. Apologies if there is one **Describe the solution you'd like** `da.dropna(['lat', 'lon], how=""all"")` produces the same as `da.dropna('lat', how=""all"").dropna('lon', how=""all"")`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4952/reactions"", ""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1013881639,I_kwDOAMm_X848bpsn,5836,to_zarr returns ,17162724,closed,0,,,2,2021-10-02T01:53:27Z,2023-03-12T15:59:14Z,2023-03-12T15:59:13Z,CONTRIBUTOR,,,," **What happened**: When doing to zarr it returns `` **What you expected to happen**: Returns None. Same behaviour as pandas.to_parquet() **Minimal Complete Verifiable Example**: ```python import xarray as xr ds = xr.tutorial.open_dataset(""air_temperature"") ds.to_zarr(""ds.zarr"") ds.to_dataframe().to_parquet(""ds.parquet"") ``` **Anything else we need to know?**: **Environment**:
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.9.7 | packaged by conda-forge | (default, Sep 23 2021, 07:31:23) [Clang 11.1.0 ] python-bits: 64 OS: Darwin OS-release: 20.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.8.0 xarray: 0.19.0 pandas: 1.3.3 numpy: 1.21.2 scipy: 1.7.1 netCDF4: 1.5.7 pydap: installed h5netcdf: 0.11.0 h5py: 3.3.0 Nio: None zarr: 2.10.0 cftime: 1.5.0 nc_time_axis: None PseudoNetCDF: None rasterio: 1.2.6 cfgrib: 0.9.9.0 iris: None bottleneck: 1.3.2 dask: 2021.09.1 distributed: 2021.09.1 matplotlib: 3.4.3 cartopy: 0.20.0 seaborn: 0.11.2 numbagg: None pint: 0.17 setuptools: 58.0.2 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 7.27.0 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5836/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1319991672,PR_kwDOAMm_X848M442,6841,DOC: link DataArray.expand_dims to Dataset.expand_dims,17162724,closed,0,,,1,2022-07-27T18:43:27Z,2022-07-27T20:20:35Z,2022-07-27T20:20:35Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6841," - [ ] Closes #xxxx - [ ] Tests added - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` Help navigation in the docs of going between the DataArray method and the Dataset method.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6841/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1295056717,PR_kwDOAMm_X8465TI4,6755,add axis arg to to_array,17162724,closed,0,,,0,2022-07-06T03:04:55Z,2022-07-06T17:52:59Z,2022-07-06T17:52:59Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6755," - [x] Closes #6754 - [x] Tests added - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6755/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1294978633,I_kwDOAMm_X85NL85J,6754,to_array to create a dimension as last axis,17162724,closed,0,,,6,2022-07-06T01:28:50Z,2022-07-06T17:12:25Z,2022-07-06T17:12:25Z,CONTRIBUTOR,,,,"### Is your feature request related to a problem? I do `ds.to_array(dim=""variable"").transpose(""latitude"", ""longitude"", ""variable"")`. I would like to avoid the extra transpose call. ### Describe the solution you'd like `ds.to_array(dim=""variable"", new_axis=""last"")` where `new_axis` is `Literal[""first"", ""last""] = ""first""` or `ds.to_array(dim=""variable"", new_axis=0)` where `new_axis` is `Literal[0, -1] = 0` code to change: https://github.com/pydata/xarray/blob/main/xarray/core/dataset.py#L5770 ### Describe alternatives you've considered _No response_ ### Additional context I imagine `new_axis` could be of type int to place the new axis where you would like but the proposal above may be a good first step. For reference, i'm doing deep learning and want to shape the data as width (latitude), height (longitude), channel (feature)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6754/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1270159911,PR_kwDOAMm_X845mZtV,6696,DOC: note of how `chunks` can be defined,17162724,closed,0,,,1,2022-06-14T02:12:56Z,2022-06-20T16:51:31Z,2022-06-20T16:51:31Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6696,"Some tiny overlap with https://github.com/pydata/xarray/pull/6542. I feel this minor changes helps it easy to navigate to other parts of the docs. Some motivation behind the PR here: https://github.com/pydata/xarray/discussions/6697 Note: i've forgotten the syntax for how to create a link to https://xarray.pydata.org/en/stable/generated/xarray.Dataset.chunk.html","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6696/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1223527885,I_kwDOAMm_X85I7Y3N,6567,"cftime_range missing freq (""W"") - Invalid frequency string provided",17162724,open,0,,,0,2022-05-03T01:08:01Z,2022-05-03T01:51:59Z,,CONTRIBUTOR,,,,"### Is your feature request related to a problem? I tend to use xr.cftime_range as a drop in for pandas.date_range. My only reason here is I may able to shave a line and not import pandas. In doing `xr.cftime_range(""2020-01-01"", periods=1, freq=""W"")` I get ``` AttributeError Traceback (most recent call last) /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftime_offsets.py in to_offset(freq) 729 try: --> 730 freq_data = re.match(_PATTERN, freq).groupdict() 731 except AttributeError: AttributeError: 'NoneType' object has no attribute 'groupdict' During handling of the above exception, another exception occurred: ValueError Traceback (most recent call last) /tmp/ipykernel_549/3579801314.py in ----> 1 xr.cftime_range(""2020-01-01"", periods=1, freq=""W"") /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftime_offsets.py in cftime_range(start, end, periods, freq, normalize, name, closed, calendar) 1041 dates = _generate_linear_range(start, end, periods) 1042 else: -> 1043 offset = to_offset(freq) 1044 dates = np.array(list(_generate_range(start, end, periods, offset))) 1045 /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftime_offsets.py in to_offset(freq) 730 freq_data = re.match(_PATTERN, freq).groupdict() 731 except AttributeError: --> 732 raise ValueError(""Invalid frequency string provided"") 733 734 freq = freq_data[""freq""] ValueError: Invalid frequency string provided ``` In pandas I get ``` >>> pd.date_range(""2020-01-01"", periods=1, freq=""W"") DatetimeIndex(['2020-01-05'], dtype='datetime64[ns]', freq='W-SUN') ``` ### Describe the solution you'd like _No response_ ### Describe alternatives you've considered Closing this issue as outside of scope ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6567/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 903922477,MDU6SXNzdWU5MDM5MjI0Nzc=,5386,"Add xr.open_dataset(""file.tif"", engine=""rasterio"") to docs",17162724,closed,0,,,1,2021-05-27T15:39:29Z,2022-04-09T03:15:45Z,2022-04-09T03:15:45Z,CONTRIBUTOR,,,,"Kind of related to https://github.com/pydata/xarray/issues/4697 I see https://corteva.github.io/rioxarray/stable/getting_started/getting_started.html#rioxarray shows `ds = xarray.open_dataset(""file.tif"", engine=""rasterio"")` This could be added to https://xarray.pydata.org/en/latest/user-guide/io.html#rasterio ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5386/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1094753761,I_kwDOAMm_X85BQJ3h,6143,"[FEATURE]: allow ""now"" as input to cftime_range",17162724,open,0,,,2,2022-01-05T21:21:42Z,2022-01-07T23:23:02Z,,CONTRIBUTOR,,,,"### Is your feature request related to a problem? For better or worse, I use `xr.cftime_range` interchangably with `pd.date_range`, mostly it is saves my importing pandas. Pandas allows a string named ""now"" to be parsed in `pd.date_range` ``` pd.date_range(""2000-01-01"", ""now"", freq=""H"") ``` It would be nice if `xr.cftime_range` could do the same (see additional context for Traceback). ### Describe the solution you'd like Something like assert `pd.date_range(""2000-01-01"", ""now"", freq=""H"")` == `xr.cftime_range(""2000-01-01"", ""now"", freq=""H"")` ### Describe alternatives you've considered Close as not within the scope of `xr.cftime_range` ### Additional context `xr.cftime_range(""2000-01-01"", ""now"", freq=""H"")` ```python-traceback ValueError Traceback (most recent call last) /tmp/ipykernel_15451/3415490845.py in ----> 1 xr.cftime_range(""2000-01-01"", ""now"", freq=""H"") /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftime_offsets.py in cftime_range(start, end, periods, freq, normalize, name, closed, calendar) 982 start = _maybe_normalize_date(start, normalize) 983 if end is not None: --> 984 end = to_cftime_datetime(end, calendar) 985 end = _maybe_normalize_date(end, normalize) 986 /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftime_offsets.py in to_cftime_datetime(date_str_or_date, calendar) 697 ""a calendar type must be provided"" 698 ) --> 699 date, _ = _parse_iso8601_with_reso(get_date_type(calendar), date_str_or_date) 700 return date 701 elif isinstance(date_str_or_date, cftime.datetime): /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftimeindex.py in _parse_iso8601_with_reso(date_type, timestr) 125 126 default = date_type(1, 1, 1) --> 127 result = parse_iso8601_like(timestr) 128 replace = {} 129 /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/coding/cftimeindex.py in parse_iso8601_like(datetime_string) 115 if match: 116 return match.groupdict() --> 117 raise ValueError( 118 f""no ISO-8601 or cftime-string-like match for string: {datetime_string}"" 119 ) ValueError: no ISO-8601 or cftime-string-like match for string: now ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6143/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1076992588,PR_kwDOAMm_X84vr6rK,6067,DOC: fix sphinx warning,17162724,closed,0,,,1,2021-12-10T15:57:59Z,2021-12-11T04:30:51Z,2021-12-11T04:30:46Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6067," - [ ] Closes #xxxx - [ ] Tests added - [ ] Passes `pre-commit run --all-files` - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6067/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1076986833,PR_kwDOAMm_X84vr5fq,6066,DOC: cd command in build documentation,17162724,closed,0,,,1,2021-12-10T15:51:49Z,2021-12-11T00:41:37Z,2021-12-11T00:41:37Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6066,"In the section above (http://xarray.pydata.org/en/stable/contributing.html#how-to-build-the-xarray-documentation) i.e. build the env, it assumes you are in the root directory. This holds that assumptions and makes it easier for a user to copy-paste to build the docs","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6066/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1076957492,PR_kwDOAMm_X84vrzcq,6065,DOC: add link to Developers Meeting,17162724,closed,0,,,1,2021-12-10T15:21:30Z,2021-12-10T21:27:55Z,2021-12-10T21:27:55Z,CONTRIBUTOR,,0,pydata/xarray/pulls/6065,"Related to https://github.com/pydata/xarray/issues/4001#issuecomment-779556647 ![Screen Shot 2021-12-10 at 2 00 12 PM](https://user-images.githubusercontent.com/17162724/145627611-e74b1f7d-9fa5-43a1-92e3-7dfaa5cb3964.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/6065/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1043695805,PR_kwDOAMm_X84uCrMw,5932,DOC: add names of missing contributors to 0.20.0,17162724,closed,0,,,2,2021-11-03T14:47:15Z,2021-11-03T15:17:42Z,2021-11-03T15:17:42Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5932,"When I worked on https://github.com/pydata/xarray/pull/5615 I did so with a couple of team mates. I added them in the commit message: https://github.com/pydata/xarray/pull/5615/commits/67ccce2f9b31eebf38fffb5dc86b90810299111c Want to make sure they get a shout out here :)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5932/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 1015260231,I_kwDOAMm_X848g6RH,5838,update ValueError for open_mfdataset for wild-card matching,17162724,closed,0,,,0,2021-10-04T14:32:45Z,2021-10-11T02:58:47Z,2021-10-11T02:58:47Z,CONTRIBUTOR,,,,"**What happened**: Took `engine=""zarr""` out of my `open_mfdataset` call and got ``` ValueError Traceback (most recent call last) /tmp/ipykernel_24527/4212238570.py in 3 TqdmCallback(desc=""dask tasks"").register() 4 ----> 5 ds = xr.open_mfdataset( 6 ""s3://era5-pds/zarr/*/*/data/air_temperature_at_2_metres.zarr"", 7 parallel=True, /opt/userenvs/ray.bell/main/lib/python3.9/site-packages/xarray/backends/api.py in open_mfdataset(paths, chunks, concat_dim, compat, preprocess, engine, data_vars, coords, combine, parallel, join, attrs_file, combine_attrs, **kwargs) 860 paths = [fs.get_mapper(path) for path in paths] 861 elif is_remote_uri(paths): --> 862 raise ValueError( 863 ""cannot do wild-card matching for paths that are remote URLs: "" 864 ""{!r}. Instead, supply paths as an explicit list of strings."".format( ValueError: cannot do wild-card matching for paths that are remote URLs: 's3://era5-pds/zarr/*/*/data/air_temperature_at_2_metres.zarr'. Instead, supply paths as an explicit list of strings. ``` **What you expected to happen**: Give a suggestion that this can be fixed if `engine=""zarr""` **Minimal Complete Verifiable Example**: ```python import xarray as xr ds = xr.open_mfdataset( ""s3://era5-pds/zarr/2020/1*/data/eastward_wind_at_10_metres.zarr"", backend_kwargs=dict(storage_options={""anon"": True}), ) Traceback (most recent call last): File """", line 1, in File ""/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/backends/api.py"", line 862, in open_mfdataset raise ValueError( ValueError: cannot do wild-card matching for paths that are remote URLs: 's3://era5-pds/zarr/2020/1*/data/eastward_wind_at_10_metres.zarr'. Instead, supply paths as an explicit list of strings. ``` ```python ds = xr.open_mfdataset( ""s3://era5-pds/zarr/2020/1*/data/eastward_wind_at_10_metres.zarr"", backend_kwargs=dict(storage_options={""anon"": True}), engine=""zarr"", ) # works ``` **Anything else we need to know?**: message here: https://github.com/pydata/xarray/blob/main/xarray/backends/api.py#L861 **Environment**:
Output of xr.show_versions()
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5838/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 1022161430,PR_kwDOAMm_X84s_dvN,5851,update ValueError message for wild card matching for remote files,17162724,closed,0,,,1,2021-10-11T01:31:18Z,2021-10-11T02:58:47Z,2021-10-11T02:58:47Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5851," - [x] Closes #5838 - [ ] Tests added - [ ] Passes `pre-commit run --all-files` - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5851/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 997343167,PR_kwDOAMm_X84ry9KI,5797,DOC: sortby see also,17162724,closed,0,,,3,2021-09-15T17:35:35Z,2021-09-15T20:16:36Z,2021-09-15T19:49:07Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5797," - [x] Closes my brain dump in https://github.com/pydata/xarray/discussions/5795 - [ ] Tests added - [ ] Passes `pre-commit run --all-files` - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5797/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 946543524,MDExOlB1bGxSZXF1ZXN0NjkxNjk5NDMz,5615,add storage_options arg to to_zarr,17162724,closed,0,,,7,2021-07-16T19:26:54Z,2021-08-21T23:19:12Z,2021-08-21T22:52:18Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5615," - [ ] Tests added - [x] Passes `pre-commit run --all-files` - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` ### What does this PR do? Adds a `storage_options` arg to `to_zarr`. ### What is the `storage_options` arg? The `storage_options` arg is used throughout the pydata ecosystem where you can write a file to cloud storage. Such as: - **pandas**: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_parquet.html?highlight=to_parquet - **dask.dataframe**: https://docs.dask.org/en/latest/generated/dask.dataframe.to_parquet.html - **dask.array**: https://docs.dask.org/en/latest/generated/dask.array.to_zarr.html#dask.array.to_zarr It allows you to write to a prd account or a dev account for example. It is can be used currently in xarray when reading data via `open_dataset` e.g. `xr.open_dataset(..., backend_kwargs=dict(storage_options=storage_options)` ### Why is it needed? When working in cloud based environments you storage options are often automatically created. I recently found when doing things in parallel dask can hiccup on finding the credentials (see https://dask.slack.com/archives/C02282J3Q6Q/p1626201210115500) passing the storage options as an arg gave it some stability. I imagine this could help when writing multiple zarr stores to cloud storage in parallel (e.g. delayed). This PR also brings similar functionality of `pandas.DataFrame.to_parquet` and `dask.array.to_zarr` to xarray. ### How is this tested? Not obvious how to test. e.g. no `storage_options` test in dask.array https://github.com/dask/dask/blob/main/dask/array/tests/test_array_core.py Pandas has more extensive tests https://github.com/pandas-dev/pandas/blob/master/pandas/tests/io/test_fsspec.py ### Background: The idea for this PR is discussed here: https://github.com/pydata/xarray/discussions/5601 cc. @martindurant ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 962974408,MDU6SXNzdWU5NjI5NzQ0MDg=,5679,DOC: remove suggestion to install pytest-xdist in docs,17162724,closed,0,,,1,2021-08-06T18:58:47Z,2021-08-19T22:16:19Z,2021-08-19T22:16:19Z,CONTRIBUTOR,,,,"In http://xarray.pydata.org/en/stable/contributing.html#running-the-test-suite The suggestion is > Using pytest-xdist, one can speed up local testing on multicore machines. To use this feature, you will need to install pytest-xdist via: pip install pytest-xdist Then, run pytest with the optional -n argument: pytest xarray -n 4 pytest-xdist is in the environment (https://github.com/pydata/xarray/blob/main/ci/requirements/environment.yml#L39) no need to reinstall ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5679/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 943922554,MDExOlB1bGxSZXF1ZXN0Njg5NDgzNTEz,5599,DOC: add open_mfdataset in See Also,17162724,closed,0,,,2,2021-07-14T00:21:06Z,2021-07-14T04:26:14Z,2021-07-14T04:00:07Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5599,Makes life a little easier when navigating the docs,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5599/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 942198905,MDU6SXNzdWU5NDIxOTg5MDU=,5596,permission error on writing zarr to s3,17162724,closed,0,,,1,2021-07-12T15:48:07Z,2021-07-12T16:01:47Z,2021-07-12T16:01:46Z,CONTRIBUTOR,,,,"Note: for upstream issue see https://github.com/dask/dask/issues/7887 **What happened**: `ds.to_zarr(""s3://BUCKET/file.zarr"")` gives **What you expected to happen**: Finds AWS credentials and writes zarr object to AWS S3 cloud storage. **Minimal Complete Verifiable Example**: ```python import xarray as xr ds = xr.tutorial.open_dataset(""air_temperature"").isel(time=0) ds.to_zarr(""s3://BUCKET/file.zarr"") --------------------------------------------------------------------------- ClientError Traceback (most recent call last) ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/s3fs/core.py in _call_s3(self, method, *akwarglist, **kwargs) 245 try: --> 246 out = await method(**additional_kwargs) 247 return out ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/aiobotocore/client.py in _make_api_call(self, operation_name, api_params) 153 error_class = self.exceptions.from_code(error_code) --> 154 raise error_class(parsed_response, operation_name) 155 else: ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied The above exception was the direct cause of the following exception: PermissionError Traceback (most recent call last) ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/zarr/storage.py in __setitem__(self, key, value) 1110 self.fs.rm(path, recursive=True) -> 1111 self.map[key] = value 1112 self.fs.invalidate_cache(self.fs._parent(path)) ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/fsspec/mapping.py in __setitem__(self, key, value) 151 self.fs.mkdirs(self.fs._parent(key), exist_ok=True) --> 152 self.fs.pipe_file(key, maybe_convert(value)) 153 ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/fsspec/asyn.py in wrapper(*args, **kwargs) 86 self = obj or args[0] ---> 87 return sync(self.loop, func, *args, **kwargs) 88 ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/fsspec/asyn.py in sync(loop, func, timeout, *args, **kwargs) 67 if isinstance(result[0], BaseException): ---> 68 raise result[0] 69 return result[0] ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/fsspec/asyn.py in _runner(event, coro, result, timeout) 23 try: ---> 24 result[0] = await coro 25 except Exception as ex: ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/s3fs/core.py in _pipe_file(self, path, data, chunksize, **kwargs) 865 if size < min(5 * 2 ** 30, 2 * chunksize): --> 866 return await self._call_s3( 867 ""put_object"", Bucket=bucket, Key=key, Body=data, **kwargs ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/s3fs/core.py in _call_s3(self, method, *akwarglist, **kwargs) 264 err = e --> 265 raise translate_boto_error(err) 266 PermissionError: Access Denied The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) /var/folders/rf/26llfhwd68x7cftb1z3h000w0000gp/T/ipykernel_10507/3272073269.py in 1 ds = xr.tutorial.open_dataset(""air_temperature"").isel(time=0) ----> 2 ds.to_zarr(""s3://BUCKET/file.zarr"") ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/dataset.py in to_zarr(self, store, chunk_store, mode, synchronizer, group, encoding, compute, consolidated, append_dim, region, safe_chunks) 1920 encoding = {} 1921 -> 1922 return to_zarr( 1923 self, 1924 store=store, ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/xarray/backends/api.py in to_zarr(dataset, store, chunk_store, mode, synchronizer, group, encoding, compute, consolidated, append_dim, region, safe_chunks) 1432 ) 1433 -> 1434 zstore = backends.ZarrStore.open_group( 1435 store=store, 1436 mode=mode, ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/xarray/backends/zarr.py in open_group(cls, store, mode, synchronizer, group, consolidated, consolidate_on_close, chunk_store, storage_options, append_dim, write_region, safe_chunks) 336 zarr_group = zarr.open_consolidated(store, **open_kwargs) 337 else: --> 338 zarr_group = zarr.open_group(store, **open_kwargs) 339 return cls( 340 zarr_group, consolidate_on_close, append_dim, write_region, safe_chunks ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/zarr/hierarchy.py in open_group(store, mode, cache_attrs, synchronizer, path, chunk_store, storage_options) 1183 raise ContainsGroupError(path) 1184 else: -> 1185 init_group(store, path=path, chunk_store=chunk_store) 1186 1187 # determine read only status ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/zarr/storage.py in init_group(store, overwrite, path, chunk_store) 486 487 # initialise metadata --> 488 _init_group_metadata(store=store, overwrite=overwrite, path=path, 489 chunk_store=chunk_store) 490 ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/zarr/storage.py in _init_group_metadata(store, overwrite, path, chunk_store) 513 meta = dict() # type: ignore 514 key = _path_to_prefix(path) + group_meta_key --> 515 store[key] = encode_group_metadata(meta) 516 517 ~/opt/miniconda3/envs/main/lib/python3.9/site-packages/zarr/storage.py in __setitem__(self, key, value) 1112 self.fs.invalidate_cache(self.fs._parent(path)) 1113 except self.exceptions as e: -> 1114 raise KeyError(key) from e 1115 1116 def __delitem__(self, key): KeyError: '.zgroup' ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5596/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 931798746,MDExOlB1bGxSZXF1ZXN0Njc5MjM2Mzg3,5547,DOC: typo,17162724,closed,0,,,2,2021-06-28T17:27:25Z,2021-06-28T18:11:52Z,2021-06-28T17:43:19Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5547,"missing code formatting on this https://xarray.pydata.org/en/latest/generated/xarray.open_dataset.html","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5547/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 909046909,MDExOlB1bGxSZXF1ZXN0NjU5Njc1MzIw,5427,DOC: zarr note on encoding,17162724,closed,0,,,4,2021-06-02T03:27:15Z,2021-06-21T18:14:15Z,2021-06-21T17:47:47Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5427,"It took me a while to realize this so I wanted to make a note in the docs for myself in the future and others uses. I was playing with pint-xarray and I couldn't work out why I couldn't round trip a file (https://github.com/xarray-contrib/pint-xarray/issues/109). It turns out the DataArray had an enconding attribute which was being used (and overriding my unit conversion). In addition, I was also scratching my head for a while as to where my zarr store was getting chunks from which didn't correspomd to my numpy array (https://github.com/pydata/xarray/discussions/5407). This PR just makes a note of the importance of encoding attribute of DataArray(s) when writing to a zarr store. docs page change is here: https://xray--5427.org.readthedocs.build/en/5427/generated/xarray.Dataset.to_zarr.html?highlight=zarr#xarray.Dataset.to_zarr","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5427/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 809630390,MDU6SXNzdWU4MDk2MzAzOTA=,4916,Feature request: add opendap/netcdf authorization to backend_kwargs in open_[mf]dataset,17162724,open,0,,,1,2021-02-16T20:38:16Z,2021-06-18T15:43:48Z,,CONTRIBUTOR,,,,"This is a paraphrase of a SO Q: https://stackoverflow.com/questions/66178846/read-in-authorized-opendap-url-using-xarray **Is your feature request related to a problem? Please describe.** I would like to use xarray to read data stored at UCAR's Research Data Archive. To access data you have to create an account with a username and password. **Describe the solution you'd like** ``` url = """"https://rda.ucar.edu/thredds/dodsC/files/g/ds084.1/2020/20200101/gfs.0p25.2020010100.f000.grib2"" ds = xr.open_dataset(url, backend_kwargs={""auth"": (""username"", ""password"")}) ``` **Describe alternatives you've considered** You can read this file by create dot files in your home directory (https://stackoverflow.com/a/66179413/6046019) **Additional context** The idea of ""auth"" comes from siphon (https://unidata.github.io/siphon/latest/examples/Basic_Usage.html#sphx-glr-examples-basic-usage-py). ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4916/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 923307166,MDExOlB1bGxSZXF1ZXN0NjcyMDg2MzM2,5479,DOC: add combine_by_coords in merge see also,17162724,closed,0,,,3,2021-06-17T01:46:31Z,2021-06-17T16:34:55Z,2021-06-17T16:04:24Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5479,Keewis mentioned other options to `merge` are `concat` and `combine_by_coords` (https://github.com/pydata/xarray/discussions/5462#discussioncomment-861492). This PR adds `combine_by_coords` in the see also docstring of `merge`,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5479/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 830507003,MDU6SXNzdWU4MzA1MDcwMDM=,5028,Saving zarr to remote location lower cases all data_vars,17162724,closed,0,,,5,2021-03-12T21:40:37Z,2021-06-17T12:53:22Z,2021-06-15T21:37:15Z,CONTRIBUTOR,,,," **What happened**: I saved a zarr store to a remote location (s3) and read it again and realized the name of the data variables (DataArrays) had reverted to lower case **What you expected to happen**: This does not happen when you save the zarr store locally **Minimal Complete Verifiable Example**: ```python ds = xr.tutorial.open_dataset(""air_temperature"") ds = ds.rename({'air': ""AIR""}) # Save to local ds.to_zarr(""ds.zarr"") # Save to remote ds.to_zarr(""s3://BUCKET/ds.zarr"") # Read local xr.open_dataset(""ds.zarr"", engine=""zarr"").data_vars #Data variables: # AIR (time, lat, lon) float32 ... # Read remote xr.open_dataset(""s3://BUCKET/ds.zarr"", engine=""zarr"").data_vars #Data variables: # air (time, lat, lon) float32 ... ``` **Anything else we need to know?**: **Environment**:
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-1009-aws machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.17.0 pandas: 1.2.3 numpy: 1.20.1 scipy: 1.5.3 netCDF4: 1.5.6 pydap: installed h5netcdf: 0.10.0 h5py: 3.1.0 Nio: None zarr: 2.6.1 cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: 1.2.1 cfgrib: 0.9.8.5 iris: None bottleneck: 1.3.2 dask: 2021.03.0 distributed: 2021.03.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: 0.11.1 numbagg: None pint: 0.16.1 setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: 6.2.2 IPython: 7.21.0 sphinx: None ​
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5028/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 593029940,MDU6SXNzdWU1OTMwMjk5NDA=,3929,Feature request xarray.Dataset.from_dask_dataframe,17162724,open,0,,,7,2020-04-03T02:18:35Z,2021-05-19T03:52:00Z,,CONTRIBUTOR,,,,The method [`xarray.Dataset.to_dask_dataframe`](http://xarray.pydata.org/en/stable/generated/xarray.Dataset.to_dask_dataframe.html) exists and would like to make a feature request for the opposite `xarray.Dataset.from_dask_dataframe`. The conversation started over at `dask` but it was suggested by @shoyer to implement here first https://github.com/dask/dask/issues/6058,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3929/reactions"", ""total_count"": 5, ""+1"": 5, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 874166254,MDExOlB1bGxSZXF1ZXN0NjI4NzgyNzIy,5249,DOC: add query examples,17162724,closed,0,,,2,2021-05-03T01:27:28Z,2021-05-03T04:30:20Z,2021-05-03T04:30:20Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5249,"Add query examples in the docstrings. Relevent rtd buld links: - https://xray--5249.org.readthedocs.build/en/5249/generated/xarray.Dataset.query.html?highlight=query#xarray.Dataset.query - https://xray--5249.org.readthedocs.build/en/5249/generated/xarray.DataArray.query.html?highlight=query#xarray.DataArray.query","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5249/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 860274641,MDExOlB1bGxSZXF1ZXN0NjE3MjA5NDc2,5176,DOC: fix link to io.zarr,17162724,closed,0,,,3,2021-04-17T00:50:57Z,2021-04-19T19:10:19Z,2021-04-19T18:13:36Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5176,Fixes rendering in https://github.com/pydata/xarray/pull/5163,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5176/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 858115471,MDU6SXNzdWU4NTgxMTU0NzE=,5160,DOC: add dropdown for User Guide -> Reading and Writing Files,17162724,open,0,,,2,2021-04-14T17:39:17Z,2021-04-19T02:44:58Z,,CONTRIBUTOR,,,,"The docs current looks likes below. There are a lot of useful sections in Reading and Writing Files (http://xarray.pydata.org/en/latest/user-guide/io.html). Not sure if it's possible to add a drop down arrow (a third level) to see the sections from the table of contents on the left ![out](https://user-images.githubusercontent.com/17162724/114754578-bb340a00-9d26-11eb-8266-f381bd98ea4f.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5160/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 830515225,MDU6SXNzdWU4MzA1MTUyMjU=,5029,ENH: allow dimension to be collapsable in the html_repr,17162724,open,0,,,0,2021-03-12T21:45:55Z,2021-04-19T02:40:44Z,,CONTRIBUTOR,,,,https://github.com/pydata/xarray/discussions/5004,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5029/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 859020732,MDExOlB1bGxSZXF1ZXN0NjE2MTY4MTA4,5163,DOC: add see also for to_zarr,17162724,closed,0,,,5,2021-04-15T15:54:15Z,2021-04-16T23:58:54Z,2021-04-16T15:30:12Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5163," - [x] Closes #5159 - [ ] Tests added - [ ] Passes `pre-commit run --all-files` - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst` - [ ] New functions/methods are listed in `api.rst`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5163/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 858096239,MDU6SXNzdWU4NTgwOTYyMzk=,5159,DOC: link to_zarr docs to io section on zarr,17162724,closed,0,,,0,2021-04-14T17:12:27Z,2021-04-16T15:30:12Z,2021-04-16T15:30:12Z,CONTRIBUTOR,,,,"the to_zarr docs http://xarray.pydata.org/en/latest/generated/xarray.Dataset.to_zarr.html could add a ""See Also"" and point to http://xarray.pydata.org/en/latest/user-guide/io.html?highlight=zarr#zarr","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5159/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 822250814,MDU6SXNzdWU4MjIyNTA4MTQ=,4993,add xr.set_options to docs,17162724,closed,0,,,3,2021-03-04T15:41:20Z,2021-03-07T16:45:46Z,2021-03-07T16:45:46Z,CONTRIBUTOR,,,,https://github.com/pydata/xarray/issues/4992#issuecomment-790685921,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4993/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 822202759,MDU6SXNzdWU4MjIyMDI3NTk=,4992,"Feature request: xr.set_option('display.max_data_vars', N)",17162724,closed,0,,,4,2021-03-04T14:48:29Z,2021-03-04T15:14:50Z,2021-03-04T15:14:50Z,CONTRIBUTOR,,,,"idea discussed here: https://github.com/pydata/xarray/discussions/4991 Copying here the data_vars method provides a convenient ways to see the varaibles and the formatting (https://github.com/pydata/xarray/blob/master/xarray/core/dataset.py#L484) is nice. However, there are times I would look to see the full variables names not just a subset (12 is the default). Ideally there would be a setting to toggle to specify the length of the formatter akin to https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html#overview xr.set_option('display.max_data_vars', N)","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4992/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 816059432,MDExOlB1bGxSZXF1ZXN0NTc5Nzg0OTc0,4956,DOC: add example for reindex,17162724,closed,0,,,2,2021-02-25T03:56:12Z,2021-02-26T06:55:49Z,2021-02-26T06:55:48Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4956,,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4956/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 815770082,MDExOlB1bGxSZXF1ZXN0NTc5NTQ0NDMy,4949,DOC: rm np import,17162724,closed,0,,,2,2021-02-24T19:19:03Z,2021-02-25T08:46:42Z,2021-02-25T08:46:41Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4949,,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4949/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 748229907,MDU6SXNzdWU3NDgyMjk5MDc=,4598,Calling pd.to_datetime on cftime variable,17162724,closed,0,,,4,2020-11-22T12:14:27Z,2021-02-16T02:42:35Z,2021-02-16T02:42:35Z,CONTRIBUTOR,,,,"It would be nice to be able to convert cftime variables to pandas datetime to utilize the functionality there. I understand this is an upstream issue as pandas probably isn't aware of cftime. However, i'm curious if a method could be added to cftime such as .to_dataframe(). I've found `pd.to_datetime(np.datetime64(date_cf))` is the best way to do this currently. ``` import xarray as xr import numpy as np import pandas as pd date_str = '2020-01-01' date_np = np.datetime64(date_str) >>> date_np numpy.datetime64('2020-01-01') date_pd = pd.to_datetime(date_np) >>> date_pd Timestamp('2020-01-01 00:00:00') date_cf = xr.cftime_range(start=date_str, periods=1)[0] pd.to_datetime(date_cf) >>> pd.to_datetime(date_cf) Traceback (most recent call last): File """", line 1, in File ""/home/ray/local/bin/anaconda3/envs/a/lib/python3.8/site-packages/pandas/core/tools/datetimes.py"", line 830, in to_datetime result = convert_listlike(np.array([arg]), format)[0] File ""/home/ray/local/bin/anaconda3/envs/a/lib/python3.8/site-packages/pandas/core/tools/datetimes.py"", line 459, in _convert_listlike_datetimes result, tz_parsed = objects_to_datetime64ns( File ""/home/ray/local/bin/anaconda3/envs/a/lib/python3.8/site-packages/pandas/core/arrays/datetimes.py"", line 2044, in objects_to_datetime64ns result, tz_parsed = tslib.array_to_datetime( File ""pandas/_libs/tslib.pyx"", line 352, in pandas._libs.tslib.array_to_datetime File ""pandas/_libs/tslib.pyx"", line 579, in pandas._libs.tslib.array_to_datetime File ""pandas/_libs/tslib.pyx"", line 718, in pandas._libs.tslib.array_to_datetime_object File ""pandas/_libs/tslib.pyx"", line 552, in pandas._libs.tslib.array_to_datetime TypeError: is not convertible to datetime ``` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4598/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 807614965,MDU6SXNzdWU4MDc2MTQ5NjU=,4901,document example of preprocess with open_mfdataset,17162724,closed,0,,,1,2021-02-12T23:37:04Z,2021-02-13T03:21:08Z,2021-02-13T03:21:08Z,CONTRIBUTOR,,,,"@jhamman's SO answer circa 2018 helped me this week https://stackoverflow.com/a/51714004/6046019 I wonder if it's worth (not sure where) providing an example of how to use `preprocesses` with `open_mfdataset`? Add an Examples entry to the doc string? (http://xarray.pydata.org/en/latest/generated/xarray.open_mfdataset.html / https://github.com/pydata/xarray/blob/5296ed18272a856d478fbbb3d3253205508d1c2d/xarray/backends/api.py#L895) While not a small example (as the remote files are large) this is how I used it: ``` import xarray as xr import s3fs def preprocess(ds): return ds.expand_dims('time') fs = s3fs.S3FileSystem(anon=True) f1 = fs.open('s3://fmi-opendata-rcrhirlam-surface-grib/2021/02/03/00/numerical-hirlam74-forecast-MaximumWind-20210203T000000Z.grb2') f2 = fs.open('s3://fmi-opendata-rcrhirlam-surface-grib/2021/02/03/06/numerical-hirlam74-forecast-MaximumWind-20210203T060000Z.grb2') ds = xr.open_mfdataset([f1, f2], engine=""cfgrib"", preprocess=preprocess, parallel=True) ``` with one file looking like: ``` xr.open_dataset(""LOCAL_numerical-hirlam74-forecast-MaximumWind-20210203T000000Z.grb2"", engine=""cfgrib"") Dimensions: (latitude: 947, longitude: 5294, step: 55) Coordinates: time datetime64[ns] ... * step (step) timedelta64[ns] 01:00:00 ... 2 days 07:00:00 heightAboveGround int64 ... * latitude (latitude) float64 25.65 25.72 25.78 ... 89.86 89.93 90.0 * longitude (longitude) float64 -180.0 -179.9 -179.9 ... 179.9 180.0 valid_time (step) datetime64[ns] ... Data variables: fg10 (step, latitude, longitude) float32 ... Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts history: 2021-02-12T18:06:52 GRIB to CDM+CF via cfgrib-0.... ``` A smaller example could be (WIP; note I was hoping ds would concat along t but it doesn't do what I expect) ``` import numpy as np import xarray as xr f1 = xr.DataArray(np.arange(2), coords=[np.arange(2)], dims=[""a""], name=""f1"") f1 = f1.assign_coords(t=0) f1.to_dataset().to_zarr(""f1.zarr"") # What's the best way to store small files to open again with mf_dataset? csv via xarray objects? can you use open_mfdataset on pkl objects? f2 = xr.DataArray(np.arange(2), coords=[np.arange(2)], dims=[""a""], name=""f2"") f2 = f2.assign_coords(t=1) f2.to_dataset().to_zarr(""f2.zarr"") # Concat along t def preprocess(ds): return ds.expand_dims('t') ds = xr.open_mfdataset([""f1.zarr"", ""f2.zarr""], engine=""zarr"", concat_dim=""t"", preprocess=preprocess) >>> ds Dimensions: (a: 2, t: 1) Coordinates: * t (t) int64 0 * a (a) int64 0 1 Data variables: f1 (t, a) int64 dask.array f2 (t, a) int64 dask.array ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4901/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 781536092,MDExOlB1bGxSZXF1ZXN0NTUxMjQyMjI0,4778,DOC: update hyperlink for xskillscore,17162724,closed,0,,,1,2021-01-07T19:00:16Z,2021-01-07T19:47:19Z,2021-01-07T19:47:14Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4778,Update hyperlink for xskillscore,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4778/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 755626157,MDExOlB1bGxSZXF1ZXN0NTMxMzAzMTY3,4645,DOC: add examples to concat,17162724,closed,0,,,13,2020-12-02T21:33:44Z,2020-12-19T23:20:26Z,2020-12-19T23:20:22Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4645," - [x] Closes #4620 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4645/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 753037814,MDU6SXNzdWU3NTMwMzc4MTQ=,4620,Link concat info to concat doc string ,17162724,closed,0,,,1,2020-11-29T22:59:08Z,2020-12-19T23:20:22Z,2020-12-19T23:20:22Z,CONTRIBUTOR,,,,Could link http://xarray.pydata.org/en/stable/combining.html#concatenate or add the example to the concat doc string: http://xarray.pydata.org/en/stable/generated/xarray.concat.html,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4620/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 735267574,MDExOlB1bGxSZXF1ZXN0NTE0NjkwNTA3,4561,rm inplace arg to rename,17162724,closed,0,,,7,2020-11-03T12:10:12Z,2020-11-07T06:55:49Z,2020-11-07T06:55:49Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4561,"This PR follows a TypeError I got ``` ds = xr.open_dataset('https://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p25deg/Best') ds.rename({'lat': 'latitude', 'lon': 'longitude'}, inplace=True) TypeError: The `inplace` argument has been removed from xarray. You can achieve an identical effect with python's standard assignment. ``` I saw the option of the inplace in the docs http://xarray.pydata.org/en/stable/generated/xarray.Dataset.rename.html Removing this argument. TODO: - [ ] rm test_rename_inplace","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4561/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 709648652,MDU6SXNzdWU3MDk2NDg2NTI=,4466,convert n-dim array to pandas dataframe,17162724,open,0,,,3,2020-09-27T02:08:55Z,2020-09-27T19:48:42Z,,CONTRIBUTOR,,,,"I just noticed that doing ``` da = xr.DataArray(np.random.rand(10, 10, 10), dims=('a', 'b', 'c')) da.to_pandas() ``` gives ``` --------------------------------------------------------------------------- KeyError Traceback (most recent call last) ~/local/bin/anaconda3/envs/test_env/lib/python3.8/site-packages/xarray/core/dataarray.py in to_pandas(self) 2397 try: -> 2398 constructor = constructors[self.ndim] 2399 except KeyError: KeyError: 3 During handling of the above exception, another exception occurred: ValueError Traceback (most recent call last) in ----> 1 da.to_pandas() ~/local/bin/anaconda3/envs/test_env/lib/python3.8/site-packages/xarray/core/dataarray.py in to_pandas(self) 2398 constructor = constructors[self.ndim] 2399 except KeyError: -> 2400 raise ValueError( 2401 ""cannot convert arrays with %s dimensions into "" 2402 ""pandas objects"" % self.ndim ValueError: cannot convert arrays with 3 dimensions into pandas objects ``` It is worth raising a suggestion (see below)? Or could you even do `da.to_dataset(name='da').to_dataframe()` under the hood. I feel the output would be what the user expects but I imagine that is not always true ``` da.to_dataset(name='da').to_dataframe() ``` ![Screenshot from 2020-09-26 22-08-23](https://user-images.githubusercontent.com/17162724/94354011-dc603200-0044-11eb-8bcb-05e3cc1b12e0.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 709398689,MDExOlB1bGxSZXF1ZXN0NDkzNDY4MjI0,4464,update url for isort in .pre-commit-config.yaml,17162724,closed,0,,,1,2020-09-26T01:59:49Z,2020-09-26T09:45:06Z,2020-09-26T09:45:06Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4464,"When opening the current url https://github.com/timothycrosley/isort it points to a new url https://github.com/PyCQA/isort. This PR replaces the old url with the new one ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4464/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 697283733,MDU6SXNzdWU2OTcyODM3MzM=,4413,rtd DataArray.to_netcdf,17162724,closed,0,,,1,2020-09-10T00:59:03Z,2020-09-17T12:59:09Z,2020-09-17T12:59:09Z,CONTRIBUTOR,,,,"http://xarray.pydata.org/en/stable/generated/xarray.DataArray.to_netcdf.html#xarray.DataArray.to_netcdf I believe there is a way to hyperlink to `xarray.Dataset.to_netcdf`. I believe there is also a way for sphinx to render the Note section e.g. in the doc string add `.. note::`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4413/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 638080883,MDU6SXNzdWU2MzgwODA4ODM=,4151,doc: reading via cfgrib,17162724,closed,0,,,2,2020-06-13T02:37:37Z,2020-06-17T16:52:30Z,2020-06-17T16:52:30Z,CONTRIBUTOR,,,,"In the docs http://xarray.pydata.org/en/stable/io.html#grib-format-via-cfgrib Curious if `eccodes` is needed if reading a grib using using the `cfgrib` backend? Does installing `cfgrib` via conda also install the binary dependencies? https://github.com/ecmwf/cfgrib#installation cc. @alexamici ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4151/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 640100500,MDExOlB1bGxSZXF1ZXN0NDM1NTcwNzUx,4162,DOC: drop eccodes in docs,17162724,closed,0,,,2,2020-06-17T02:55:06Z,2020-06-17T16:52:30Z,2020-06-17T16:52:30Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4162," - [x] Closes #4151 - [NA] Tests added - [NA] Passes `isort -rc . && black . && mypy . && flake8` - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4162/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 605181397,MDExOlB1bGxSZXF1ZXN0NDA3NjM0Nzkx,3994,DOC: add pandas.DataFrame.to_xarray,17162724,closed,0,,,1,2020-04-23T01:54:10Z,2020-04-23T07:58:13Z,2020-04-23T07:58:09Z,CONTRIBUTOR,,0,pydata/xarray/pulls/3994," - [X] Closes #3895 - [ ] Tests added - [ ] Passes `isort -rc . && black . && mypy . && flake8` - [ ] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API Tested the docs build ![image](https://user-images.githubusercontent.com/17162724/80051253-bbb80d00-84e5-11ea-93f0-288497280738.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3994/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 588118015,MDU6SXNzdWU1ODgxMTgwMTU=,3895,xarray.Dataset.from_dataframe link to pandas.DataFrame.to_xarray,17162724,closed,0,,,3,2020-03-26T02:57:24Z,2020-04-23T07:58:09Z,2020-04-23T07:58:09Z,CONTRIBUTOR,,,,"Is it worth referencing https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_xarray.html in http://xarray.pydata.org/en/stable/generated/xarray.Dataset.from_dataframe.html? is the pandas method preferred?","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3895/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 561043452,MDU6SXNzdWU1NjEwNDM0NTI=,3756,pynio package not found,17162724,closed,0,,,1,2020-02-06T14:20:45Z,2020-03-09T14:07:03Z,2020-03-09T14:07:03Z,CONTRIBUTOR,,,,"Copied the install instructions from the docs http://xarray.pydata.org/en/stable/installing.html#instructions on my windows machine. `conda install -c conda-forge xarray cartopy pynio pseudonetcdf` Got a `PackagesNotFoundError: pynio` ![image](https://user-images.githubusercontent.com/17162724/73945181-e66ccf00-48c1-11ea-9746-e238ce48ae5c.png) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3756/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 558742452,MDU6SXNzdWU1NTg3NDI0NTI=,3742,GRIB Data Example notebook - dataset not found,17162724,closed,0,,,3,2020-02-02T19:15:55Z,2020-02-03T17:05:07Z,2020-02-03T17:05:07Z,CONTRIBUTOR,,,,"Just testing the docs on binder (good job). Noticed the ERA5-Grib-example.ipynb (https://github.com/pydata/xarray/blob/master/doc/examples/ERA5-GRIB-example.ipynb) was not working ![image](https://user-images.githubusercontent.com/17162724/73613732-7959f080-45c6-11ea-8416-254d73a4fb2f.png) ``` --------------------------------------------------------------------------- HTTPError Traceback (most recent call last) in ----> 1 ds = xr.tutorial.load_dataset('era5-2mt-2019-03-uk.grib', engine='cfgrib') /srv/conda/envs/notebook/lib/python3.8/site-packages/xarray/tutorial.py in load_dataset(*args, **kwargs) 107 open_dataset 108 """""" --> 109 with open_dataset(*args, **kwargs) as ds: 110 return ds.load() 111 /srv/conda/envs/notebook/lib/python3.8/site-packages/xarray/tutorial.py in open_dataset(name, cache, cache_dir, github_url, branch, **kws) 75 76 url = ""/"".join((github_url, ""raw"", branch, fullname)) ---> 77 urlretrieve(url, localfile) 78 url = ""/"".join((github_url, ""raw"", branch, md5name)) 79 urlretrieve(url, md5file) /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in urlretrieve(url, filename, reporthook, data) 245 url_type, path = _splittype(url) 246 --> 247 with contextlib.closing(urlopen(url, data)) as fp: 248 headers = fp.info() 249 /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context) 220 else: 221 opener = _opener --> 222 return opener.open(url, data, timeout) 223 224 def install_opener(opener): /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in open(self, fullurl, data, timeout) 529 for processor in self.process_response.get(protocol, []): 530 meth = getattr(processor, meth_name) --> 531 response = meth(req, response) 532 533 return response /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in http_response(self, request, response) 638 # request was successfully received, understood, and accepted. 639 if not (200 <= code < 300): --> 640 response = self.parent.error( 641 'http', request, response, code, msg, hdrs) 642 /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in error(self, proto, *args) 567 if http_err: 568 args = (dict, 'default', 'http_error_default') + orig_args --> 569 return self._call_chain(*args) 570 571 # XXX probably also want an abstract factory that knows when it makes /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args) 500 for handler in handlers: 501 func = getattr(handler, meth_name) --> 502 result = func(*args) 503 if result is not None: 504 return result /srv/conda/envs/notebook/lib/python3.8/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs) 647 class HTTPDefaultErrorHandler(BaseHandler): 648 def http_error_default(self, req, fp, code, msg, hdrs): --> 649 raise HTTPError(req.full_url, code, msg, hdrs, fp) 650 651 class HTTPRedirectHandler(BaseHandler): HTTPError: HTTP Error 404: Not Found ```","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3742/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 223610210,MDU6SXNzdWUyMjM2MTAyMTA=,1381,xr.where not picking up nan from another NetCDF file,17162724,closed,0,,,5,2017-04-23T03:45:24Z,2019-04-25T15:23:43Z,2019-04-25T15:23:43Z,CONTRIBUTOR,,,,"I posted this question here: http://stackoverflow.com/questions/43485347/python-xarray-copy-nan-from-one-dataarray-to-another I have uploaded the files and code to https://github.com/raybellwaves/xarray_issue.git When I originally discovered the issue I was using python version 3.6.0 (default, Jan 28 2017, 13:49:14) [GCC Intel(R) C++ gcc 4.4 mode] I tested the code with version 2.7.13 | packaged by conda-forge | (default, Mar 20 2017, 14:26:36) [GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] and I did not get the error message which I posted in the question. The code worked by the variable ws10_masked is empty","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1381/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 354803087,MDExOlB1bGxSZXF1ZXN0MjExNDc4OTAx,2387,DOC: move xskillscore to 'Extend xarray capabilities',17162724,closed,0,,,1,2018-08-28T16:13:13Z,2018-08-28T18:12:12Z,2018-08-28T18:11:56Z,CONTRIBUTOR,,0,pydata/xarray/pulls/2387,Think my package should go in the 'extend xarray capabilities' section rather than the 'geoscience' section.,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2387/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 348421086,MDExOlB1bGxSZXF1ZXN0MjA2Nzc1NDEz,2350,DOC: Add xskillscore to project lists,17162724,closed,0,,,1,2018-08-07T17:42:27Z,2018-08-07T18:05:35Z,2018-08-07T18:05:35Z,CONTRIBUTOR,,0,pydata/xarray/pulls/2350,Potentially fixes #1115 as I have a correlation function. http://xarray.pydata.org/en/stable/dask.html?highlight=spearman#automatic-parallelization probably fixes #1115 as well,"{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2350/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 344608441,MDExOlB1bGxSZXF1ZXN0MjAzOTY0Mzc1,2312,DOC: add interp example,17162724,closed,0,,,4,2018-07-25T21:10:08Z,2018-07-28T06:18:32Z,2018-07-28T06:17:22Z,CONTRIBUTOR,,0,pydata/xarray/pulls/2312," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) I wanted to add an example as googling 'xarray interp' takes me to http://xarray.pydata.org/en/stable/generated/xarray.Dataset.interp.html. However, there lots of great examples in http://xarray.pydata.org/en/stable/interpolation.html Perhaps I should edit it to: ``` Examples -------- See http://xarray.pydata.org/en/stable/interpolation.html ``` I was also thinking about adding a note about interpolating time as it came up in https://github.com/pydata/xarray/issues/2284 but not sure what would be the best way to do so. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2312/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 321917084,MDU6SXNzdWUzMjE5MTcwODQ=,2113,Rolling mean of dask array conflicting sizes for data and coordinate in rolling operation,17162724,closed,0,,,4,2018-05-10T12:40:19Z,2018-05-12T06:15:55Z,2018-05-12T06:15:55Z,CONTRIBUTOR,,,,"#### Code Sample, a copy-pastable example if possible ```python import xarray as xr remote_data = xr.open_dataarray('http://iridl.ldeo.columbia.edu/SOURCES/.Models'\ '/.SubX/.RSMAS/.CCSM4/.hindcast/.zg/dods', chunks={'L': 1, 'S': 1}) da = remote_data.isel(P=0,L=0,M=0,X=0,Y=0) da_day_clim = da.groupby('S.dayofyear').mean('S') da_day_clim2 = da_day_clim.chunk({'dayofyear': 366}) da_day_clim_smooth = da_day_clim2.rolling(dayofyear=31, center=True).mean() ``` #### Problem description Initially discussed on SO: https://stackoverflow.com/questions/50265586/xarray-rolling-mean-of-dask-array-conflicting-sizes-for-data-and-coordinate-in The rolling operation gives a ```ValueError: conflicting sizes for dimension 'dayofyear': length 351 on the data but length 366 on coordinate 'dayofyear'``` The length of 351 in the data is created in the rolling operation. Here's the full traceback: ``` ValueError Traceback (most recent call last) in () 4 da_day_clim = da.groupby('S.dayofyear').mean('S') 5 da_day_clim2 = da_day_clim.chunk({'dayofyear': 366}) ----> 6 da_day_clim_smooth = da_day_clim2.rolling(dayofyear=31, center=True).mean() ~/anaconda/envs/SubXNAO/lib/python3.6/site-packages/xarray/core/rolling.py in wrapped_func(self, **kwargs) 307 if self.center: 308 values = values[valid] --> 309 result = DataArray(values, self.obj.coords) 310 311 return result ~/anaconda/envs/SubXNAO/lib/python3.6/site-packages/xarray/core/dataarray.py in __init__(self, data, coords, dims, name, attrs, encoding, fastpath) 224 225 data = as_compatible_data(data) --> 226 coords, dims = _infer_coords_and_dims(data.shape, coords, dims) 227 variable = Variable(dims, data, attrs, encoding, fastpath=True) 228 ~/anaconda/envs/SubXNAO/lib/python3.6/site-packages/xarray/core/dataarray.py in _infer_coords_and_dims(shape, coords, dims) 79 raise ValueError('conflicting sizes for dimension %r: ' 80 'length %s on the data but length %s on ' ---> 81 'coordinate %r' % (d, sizes[d], s, k)) 82 83 if k in sizes and v.shape != (sizes[k],): ValueError: conflicting sizes for dimension 'dayofyear': length 351 on the data but length 366 on coordinate 'dayofyear' ``` #### Expected Output The rolling operation would work on the dask array as it would on the dataarray e.g. ``` import pandas as pd import xarray as xr import numpy as np dates = pd.date_range('1/1/1980', '31/12/2000', freq='D') data = np.linspace(1, len(dates), num=len(dates), dtype=np.float) da = xr.DataArray(data, coords=[dates], dims='time') da_day_clim = da.groupby('time.dayofyear').mean('time') da_day_clim_smooth = da_day_clim.rolling(dayofyear=31, center=True).mean() ``` #### Output of ``xr.show_versions()``
/Users/Ray/anaconda/envs/SubXNAO/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters INSTALLED VERSIONS ------------------ commit: None python: 3.6.5.final.0 python-bits: 64 OS: Darwin OS-release: 17.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.3 pandas: 0.22.0 numpy: 1.14.2 scipy: 1.0.1 netCDF4: 1.3.1 h5netcdf: 0.5.1 h5py: 2.7.1 Nio: None zarr: None bottleneck: 1.2.1 cyordereddict: None dask: 0.17.4 distributed: 1.21.8 matplotlib: 2.2.2 cartopy: 0.16.0 seaborn: None setuptools: 39.1.0 pip: 9.0.3 conda: None pytest: None IPython: 6.3.1 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2113/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 320007162,MDU6SXNzdWUzMjAwMDcxNjI=,2102,resample DeprecationWarning only on 1-D arrays?,17162724,closed,0,,,1,2018-05-03T17:13:55Z,2018-05-08T17:36:22Z,2018-05-08T17:36:22Z,CONTRIBUTOR,,,,"#### Code Sample, a copy-pastable example if possible ```python da = xr.DataArray(np.array([1,2,3,4], dtype=np.float).reshape(2,2), ... coords=[pd.date_range('1/1/2000', '1/2/2000', freq='D'), ... np.linspace(0,1,num=2)], ... dims=['time', 'latitude']) da.resample(freq='M', dim='time', how='mean') #/Users/Ray/anaconda/envs/rot-eof-dev-env/bin/ipython:1: DeprecationWarning: #.resample() has been modified to defer calculations. Instead of passing 'dim' and 'how=""mean"", #instead consider using .resample(time=""M"").mean() # #!/Users/Ray/anaconda/envs/rot-eof-dev-env/bin/python #Out[66]: # #array([[2., 3.]]) #Coordinates: # * time (time) datetime64[ns] 2000-01-31 # * latitude (latitude) float64 0.0 1.0 da.resample(time=""M"").mean() # #array([2.5]) #Coordinates: # * time (time) datetime64[ns] 2000-01-31 ``` #### Problem description The DeprecationWarning example seems to only work for 1d arrays as it doesn't average along any dimension. A quick fix could be to show the warning only if the DataArray/Dataset is 1D. A more thorough fix could be to wrap `.resample(time=""M"").mean()` as `.resample(freq='M', dim='time', how='mean')`??? #### Expected Output Same as `da.resample(freq='M', dim='time', how='mean')` #### Output of ``xr.show_versions()``
xr.show_versions() # Not sure about the h5py FutureWarning? /Users/Ray/anaconda/envs/rot-eof-dev-env/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters INSTALLED VERSIONS ------------------ commit: None python: 3.6.5.final.0 python-bits: 64 OS: Darwin OS-release: 17.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.3 pandas: 0.22.0 numpy: 1.14.2 scipy: 1.0.1 netCDF4: 1.3.1 h5netcdf: 0.5.1 h5py: 2.7.1 Nio: None zarr: None bottleneck: 1.2.1 cyordereddict: None dask: 0.17.2 distributed: 1.21.6 matplotlib: 2.2.2 cartopy: 0.16.0 seaborn: None setuptools: 39.0.1 pip: 9.0.3 conda: None pytest: None IPython: 6.3.1 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2102/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 319985789,MDExOlB1bGxSZXF1ZXN0MTg1NzY2MTg3,2101,DOC: Add resample e.g. Edit rolling e.g. Add groupby e.g.,17162724,closed,0,,,4,2018-05-03T16:08:48Z,2018-05-08T15:46:17Z,2018-05-08T04:23:03Z,CONTRIBUTOR,,0,pydata/xarray/pulls/2101," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) Added a `resample` example to reflect the `DeprecationWarning: .resample() has been modified to defer calculations. Instead of passing 'dim' and 'how=""mean"", instead consider using...`. There were also some missing parameters in the docs. Made a minor edit to my `rolling` example which uses the parameter `center`. I can't remember if that parameter was there last time but I think it's useful to return the time value for the middle of window. Added a `groupby` example. Not sure if this should go here. See text below for my motivation of putting an example in the docstring of the class/object. I learnt `xarray` before I learnt `pandas` so these example will hopefully be useful to other beginners as it took me a minute to get the syntax right. Whilst there are great examples of these functions in the docs, a google search of a function for example `xarray rolling` often returns the doc entry that that object. Having a little example there is helpful for me to get the syntax right. The key/value pair in a dictionary for example is very powerful and always takes me a few tries to get the entry right.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2101/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 302731264,MDExOlB1bGxSZXF1ZXN0MTczMTgwNDQz,1969,Switched broken pypip.in badges to shields.io,17162724,closed,0,,,1,2018-03-06T14:51:49Z,2018-03-06T18:24:17Z,2018-03-06T18:24:17Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1969," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) This seems to be happening to a lot of projects as highlighted on the recent list of people who have cited this issue https://github.com/badges/shields/issues/716. Here is a project with a recent commit to resolve this which explains the issue better that I can https://github.com/landscapeio/pylint-celery/pull/7","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1969/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 299210082,MDExOlB1bGxSZXF1ZXN0MTcwNjQ4NzM3,1934,DOC: Add contributing section in README.rst and README.rst in doc/,17162724,closed,0,,,3,2018-02-22T03:30:26Z,2018-02-23T19:55:47Z,2018-02-23T19:55:47Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1934," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) These were inspired by a discussion on the mailing list: https://groups.google.com/forum/#!topic/xarray/FyCFkuWbFSk ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1934/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 297331231,MDExOlB1bGxSZXF1ZXN0MTY5Mjg2NDgz,1911,DOC: Removed a line in setting up toy weather data,17162724,closed,0,,,2,2018-02-15T05:11:42Z,2018-02-15T05:29:52Z,2018-02-15T05:29:28Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1911," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1911/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 279141739,MDExOlB1bGxSZXF1ZXN0MTU2MzAwNDM2,1758,DOC: Added parameter list to open_dataarray,17162724,closed,0,,,3,2017-12-04T20:41:09Z,2018-01-19T05:14:57Z,2018-01-19T05:13:51Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1758,"Closes #1757 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1758/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 278743801,MDU6SXNzdWUyNzg3NDM4MDE=,1757,"open_dataarray docs still contains *args, **kwargs",17162724,closed,0,,,4,2017-12-03T04:36:48Z,2018-01-19T05:13:51Z,2018-01-19T05:13:51Z,CONTRIBUTOR,,,,"I noticed the [open_dataset](http://xarray.pydata.org/en/stable/generated/xarray.open_dataset.html) docs provides the full input parameters whereas, [open_dataarray](http://xarray.pydata.org/en/stable/generated/xarray.open_dataarray.html) docs still lists `*args, **kwargs` If you point me to where this is created I can change it, if the change is preferred","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1757/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 283410571,MDExOlB1bGxSZXF1ZXN0MTU5MzAzNDA3,1794,DOC: added sortby example,17162724,closed,0,,,1,2017-12-20T00:16:24Z,2017-12-20T01:21:06Z,2017-12-20T01:21:04Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1794," - [NA] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes) - [NA] Tests added (for all bug fixes or enhancements) - [NA] Tests passed (for all non-documentation changes) - [NA] Passes ``git diff upstream/master **/*py | flake8 --diff`` (remove if you did not edit any Python files) - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later) ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1794/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 269224520,MDExOlB1bGxSZXF1ZXN0MTQ5Mjc4MDIy,1665,DOC: added apply example to core/dataarray.py,17162724,closed,0,,,2,2017-10-27T20:21:49Z,2017-10-27T23:17:54Z,2017-10-27T23:17:51Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1665," - [NA] Closes #xxxx - [NA] Tests added / passed - [ ] Passes ``git diff upstream/master | flake8 --diff`` - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API I added an example to apply as I just learnt about it from my Q on stackoverflow (https://stackoverflow.com/questions/46980823/python-xarray-compute-the-absolute-values-of-a-dataset/46981027#46981027). Also my first time using git and making edits on the command line rather than the web browser. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1665/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 258267613,MDExOlB1bGxSZXF1ZXN0MTQxNDY3MTc1,1574,DOC: Added examples to rolling.py,17162724,closed,0,,,4,2017-09-17T00:14:41Z,2017-10-27T20:15:39Z,2017-09-21T01:46:11Z,CONTRIBUTOR,,0,pydata/xarray/pulls/1574,"This idea for this PR was discussed here: https://stackoverflow.com/questions/45992983/python-xarray-rolling-mean-example I wasn't able to drop the nans using min_periods so I used the dropna method. If there is a more eloquent method to my example feel free to point me to it. - [NA] Closes #xxxx - [NA] Tests added / passed - [ ] Passes ``git diff upstream/master | flake8 --diff`` - [NA] Fully documented, including `whats-new.rst` for all changes and `api.rst` for new API ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1574/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull