id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1021396447,PR_kwDOAMm_X84s9Wtg,5847,Display coords' units for slice plots,31376402,closed,0,,,10,2021-10-08T19:18:38Z,2021-10-30T14:06:41Z,2021-10-30T14:06:41Z,CONTRIBUTOR,,0,pydata/xarray/pulls/5847," Hi, folks! Thank you for the awesome project! Here is a minor enhancement for the plotting of slices. Before this PR: ```python import xarray as xr import numpy as np dset = xr.Dataset( data_vars=dict( var_a=((""a"", ""b""), np.array([[1,2], [3,4]])) ), coords=dict( a=(""a"", np.array([5, 6]), {""unit"": ""s""}), b=(""b"", np.array([7, 8]), {""unit"": ""s""}) ) ) _ = dset.var_a.sel(a=5).plot(marker=""o"") ``` ![download-5](https://user-images.githubusercontent.com/31376402/136613757-17ddcb8f-1287-4b7d-8fb4-ea2129e962cd.png) After this PR (note the title): ![download-4](https://user-images.githubusercontent.com/31376402/136613784-accca490-4367-4f93-978c-f61eeadaf783.png) --- I did not add any tests, since this change did not break anything I am guessing it is not something being tested. - [ ] Tests added - [x] Passes `pre-commit run --all-files` - [x] User visible changes (including notable bug fixes) are documented in `whats-new.rst`","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5847/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 819937730,MDU6SXNzdWU4MTk5Mzc3MzA=,4985,Tests fail when installing h5netcdf without netcdf4,31376402,closed,0,,,4,2021-03-02T11:42:13Z,2021-07-17T21:04:59Z,2021-07-17T21:04:59Z,CONTRIBUTOR,,,," **What happened**: Created a new conda env for xarray as described in the contribution guide. Then I installed the `h5netcdf` and run all the tests (without making any changed in the xarray repo). Tests failed due to missing dependencies apparently. **What you expected to happen**: Tests should not fail if a package is not a requirement and the backends can operate independently. **Minimal Complete Verifiable Example**: ```bash cd xarray pip install -e . pip install h5netcdf pytest-xdist py.test -n 2 . ``` ```python ======================================================================== FAILURES ========================================================================= __________________________________________________________ TestH5NetCDFAlreadyOpen.test_deepcopy __________________________________________________________ [gw0] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8 self = def test_deepcopy(self): import h5netcdf with create_tmp_file() as tmp_file: > with nc4.Dataset(tmp_file, mode=""w"") as nc: E NameError: name 'nc4' is not defined /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:2717: NameError _____________________________________________________ TestH5NetCDFAlreadyOpen.test_open_dataset_group _____________________________________________________ [gw1] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8 self = def test_open_dataset_group(self): import h5netcdf with create_tmp_file() as tmp_file: > with nc4.Dataset(tmp_file, mode=""w"") as nc: E NameError: name 'nc4' is not defined /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:2690: NameError _____________________________________________________________ test_load_single_value_h5netcdf _____________________________________________________________ [gw0] darwin -- Python 3.8.8 /usr/local/anaconda3/envs/xarray/bin/python3.8 def _get_default_engine_netcdf(): try: > import netCDF4 # noqa: F401 E ModuleNotFoundError: No module named 'netCDF4' /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:105: ModuleNotFoundError During handling of the above exception, another exception occurred: def _get_default_engine_netcdf(): try: import netCDF4 # noqa: F401 engine = ""netcdf4"" except ImportError: # pragma: no cover try: > import scipy.io.netcdf # noqa: F401 E ModuleNotFoundError: No module named 'scipy' /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:110: ModuleNotFoundError During handling of the above exception, another exception occurred: tmp_path = PosixPath('/private/var/folders/9k/k5388g_s44b_97319sslv9c40000gn/T/pytest-of-Victor/pytest-3/popen-gw0/test_load_single_value_h5netcd0') @requires_h5netcdf def test_load_single_value_h5netcdf(tmp_path): """"""Test that numeric single-element vector attributes are handled fine. At present (h5netcdf v0.8.1), the h5netcdf exposes single-valued numeric variable attributes as arrays of length 1, as oppesed to scalars for the NetCDF4 backend. This was leading to a ValueError upon loading a single value from a file, see #4471. Test that loading causes no failure. """""" ds = xr.Dataset( { ""test"": xr.DataArray( np.array([0]), dims=(""x"",), attrs={""scale_factor"": 1, ""add_offset"": 0} ) } ) > ds.to_netcdf(tmp_path / ""test.nc"") /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_backends.py:4993: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/dataset.py:1689: in to_netcdf return to_netcdf( /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:1052: in to_netcdf engine = _get_default_engine(path_or_file) /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:145: in _get_default_engine engine = _get_default_engine_netcdf() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def _get_default_engine_netcdf(): try: import netCDF4 # noqa: F401 engine = ""netcdf4"" except ImportError: # pragma: no cover try: import scipy.io.netcdf # noqa: F401 engine = ""scipy"" except ImportError: > raise ValueError( ""cannot read or write netCDF files without "" ""netCDF4-python or scipy installed"" ) E ValueError: cannot read or write netCDF files without netCDF4-python or scipy installed /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/backends/api.py:114: ValueError ==================================================================== warnings summary ===================================================================== xarray/tests/test_accessor_str.py: 18 warnings /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/duck_array_ops.py:248: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2)) xarray/tests/test_dataset.py::TestDataset::test_resample_loffset /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_dataset.py:3904: FutureWarning: 'loffset' in .resample() and in Grouper() is deprecated. >>> df.resample(freq=""3s"", loffset=""8H"") becomes: >>> from pandas.tseries.frequencies import to_offset >>> df = df.resample(freq=""3s"").mean() >>> df.index = df.index.to_timestamp() + to_offset(""8H"") ds.bar.to_series().resample(""24H"", loffset=""-12H"").mean() xarray/tests/test_dataarray.py::TestDataArray::test_resample /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/tests/test_dataarray.py:2994: FutureWarning: 'loffset' in .resample() and in Grouper() is deprecated. >>> df.resample(freq=""3s"", loffset=""8H"") becomes: >>> from pandas.tseries.frequencies import to_offset >>> df = df.resample(freq=""3s"").mean() >>> df.index = df.index.to_timestamp() + to_offset(""8H"") expected = DataArray(array.to_series().resample(""24H"", loffset=""-12H"").mean()) xarray/tests/test_dataarray.py::TestDataArray::test_polyfit[True-False] xarray/tests/test_dataarray.py::TestDataArray::test_polyfit[False-False] /Users/Victor/Documents/ProjectsDev/Qblox/xarray/xarray/core/dataset.py:6396: RuntimeWarning: overflow encountered in multiply scale = np.sqrt((lhs * lhs).sum(axis=0)) -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================= short test summary info ================================================================= FAILED xarray/tests/test_backends.py::TestH5NetCDFAlreadyOpen::test_deepcopy - NameError: name 'nc4' is not defined FAILED xarray/tests/test_backends.py::TestH5NetCDFAlreadyOpen::test_open_dataset_group - NameError: name 'nc4' is not defined FAILED xarray/tests/test_backends.py::test_load_single_value_h5netcdf - ValueError: cannot read or write netCDF files without netCDF4-python or scipy in... ============================== 3 failed, 3474 passed, 4472 skipped, 65 xfailed, 17 xpassed, 22 warnings in 66.86s (0:01:06) =============================== ``` **Anything else we need to know?**: Tests still fail when installing `scipy` only. They only pass after installing both `netcdf4` and `scipy`. **Environment**:
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: 48378c4b11c5c2672ff91396d4284743165b4fbe python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:12:38) [Clang 11.0.1 ] python-bits: 64 OS: Darwin OS-release: 18.7.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.12.0 libnetcdf: None xarray: 0.17.1.dev3+g48378c4b pandas: 1.2.2 numpy: 1.20.1 scipy: None netCDF4: None pydap: None h5netcdf: 0.10.0 h5py: 3.1.0 Nio: None zarr: None cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: 6.2.2 IPython: None sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4985/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 819967505,MDExOlB1bGxSZXF1ZXN0NTgyOTg5Mjc2,4986,Added support for numpy.bool_,31376402,closed,0,,,2,2021-03-02T12:22:17Z,2021-06-30T17:49:25Z,2021-03-12T14:19:15Z,CONTRIBUTOR,,0,pydata/xarray/pulls/4986," Hi folks! Thank you for the awesome project! Xarray rocks! We have adopted xarray fully in a new project and would appreciate a review on this PR asap. - [X] Closes #4981 - [X] Tests added - [X] Passes `pre-commit run --all-files` ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4986/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,pull 819379617,MDU6SXNzdWU4MTkzNzk2MTc=,4981,Allows numpy.bool_ attrs when writing with h5netcdf engine,31376402,closed,0,,,5,2021-03-01T22:53:04Z,2021-03-12T14:19:14Z,2021-03-12T14:19:14Z,CONTRIBUTOR,,,," **What happened**: Round trip dataset using `h5netcdf` + ` invalid_netcdf=True` fails for bool attribute due to lack of support in xarray for `numpy.bool_` (note that is not the same as `numpy.bool`) **What you expected to happen**: Attributes of `numpy.bool_` type should be supported because the `h5netcdf` has no issue with it. **Minimal Complete Verifiable Example**: ```python import h5netcdf import numpy as np import xarray as xr dset = xr.Dataset( coords={""x0"": np.array([1, 2, 3])}, data_vars={""y0"": (""x0"", np.array([5, 6, 7]))} ) dset.attrs[""my_bool_attr""] = True print(type(dset.attrs[""my_bool_attr""])) print(""\n"", dset) ``` ``` Dimensions: (x0: 3) Coordinates: * x0 (x0) int64 1 2 3 Data variables: y0 (x0) int64 5 6 7 Attributes: my_bool_attr: True ``` ```python file_name = ""my_dset.nc"" dset.to_netcdf(file_name, engine=""h5netcdf"", invalid_netcdf=True) loaded_dset = xr.load_dataset(file_name, engine=""h5netcdf"") print(type(loaded_dset.attrs[""my_bool_attr""])) print(""\n"", loaded_dset) # This should be working file_name2 = ""file_numpy_bool.nc"" loaded_dset.to_netcdf(file_name2, engine=""h5netcdf"", invalid_netcdf=True) loaded_dset2 = xr.load_dataset(file_name2, engine=""h5netcdf"") print(""\n"", loaded_dset) ``` Raises: ``` Dimensions: (x0: 3) Coordinates: * x0 (x0) int64 1 2 3 Data variables: y0 (x0) int64 5 6 7 Attributes: my_bool_attr: True --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in 6 # This should be working 7 file_name2 = ""file_numpy_bool.nc"" ----> 8 loaded_dset.to_netcdf(file_name2, engine=""h5netcdf"", invalid_netcdf=True) 9 loaded_dset2 = xr.load_dataset(file_name2, engine=""h5netcdf"") 10 /usr/local/anaconda3/envs/dclab/lib/python3.7/site-packages/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding, unlimited_dims, compute, invalid_netcdf) 1697 unlimited_dims=unlimited_dims, 1698 compute=compute, -> 1699 invalid_netcdf=invalid_netcdf, 1700 ) 1701 /usr/local/anaconda3/envs/dclab/lib/python3.7/site-packages/xarray/backends/api.py in to_netcdf(dataset, path_or_file, mode, format, group, engine, encoding, unlimited_dims, compute, multifile, invalid_netcdf) 1057 # validate Dataset keys, DataArray names, and attr keys/values 1058 _validate_dataset_names(dataset) -> 1059 _validate_attrs(dataset) 1060 1061 try: /usr/local/anaconda3/envs/dclab/lib/python3.7/site-packages/xarray/backends/api.py in _validate_attrs(dataset) 229 # Check attrs on the dataset itself 230 for k, v in dataset.attrs.items(): --> 231 check_attr(k, v) 232 233 # Check attrs on each variable within the dataset /usr/local/anaconda3/envs/dclab/lib/python3.7/site-packages/xarray/backends/api.py in check_attr(name, value) 221 if not isinstance(value, (str, Number, np.ndarray, np.number, list, tuple)): 222 raise TypeError( --> 223 f""Invalid value for attr {name!r}: {value!r} must be a number, "" 224 ""a string, an ndarray or a list/tuple of "" 225 ""numbers/strings for serialization to netCDF "" TypeError: Invalid value for attr 'my_bool_attr': True must be a number, a string, an ndarray or a list/tuple of numbers/strings for serialization to netCDF files ``` **Anything else we need to know?**: Xarray is awesome!! Thank you, guys! ❤ We are adopting it fully for a quantum computing experimental framework ([Quantify](https://quantify-quantify-core.readthedocs-hosted.com/en/latest/?badge=latest)). This issue is very critical for us at the moment 😭 For reference, the following works in `h5netcdf`: ```python with h5netcdf.File('mydata.nc', 'w', invalid_netcdf=True) as f: f.dimensions = {'x': 5} v = f.create_variable('hello', ('x',), float) v.attrs['foo'] = True print(v.attrs) print(type(v.attrs[""foo""])) with h5netcdf.File('mydata.nc', 'r') as file: v = file['hello'] print(v.attrs) print(type(v.attrs[""foo""])) with h5netcdf.File('mydata.nc', 'w', invalid_netcdf=True) as f: f.dimensions = {'x': 5} v = f.create_variable('hello', ('x',), float) v.attrs['foo'] = np.bool_(True) print(v.attrs) print(type(v.attrs[""foo""])) ``` ``` foo: True foo: True foo: True ``` **How to fix:** In `/xarray/backends/api.py:221`: ```python if not isinstance(value, (str, Number, np.ndarray, np.number, list, tuple)): raise TypeError( f""Invalid value for attr {name!r}: {value!r} must be a number, "" ""a string, an ndarray or a list/tuple of "" ""numbers/strings for serialization to netCDF "" ""files"" ) ``` add `np.bool_`: ```python if not isinstance(value, (str, Number, np.ndarray, np.number, np.bool_, list, tuple)): raise TypeError( f""Invalid value for attr {name!r}: {value!r} must be a number, "" ""a string, an ndarray or a list/tuple of "" ""numbers/strings for serialization to netCDF "" ""files"" ) ``` I did a quick test (`xarray-0.17.1.dev3+g48378c4b`) and it seems to work, but the tests (without any changes to the code) fail on my local clone of the repo so it would be more difficult to go through a PR for this. Considering that it is a single line of code, could this be deployed relatively easily into the master branch by some maintainer? That would be highly appreciated! 🥺 **Environment**:
Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 18.7.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.4 libnetcdf: None xarray: 0.17.0 pandas: 1.0.1 numpy: 1.19.3 scipy: 1.5.4 netCDF4: None pydap: None h5netcdf: 0.10.0 h5py: 2.10.0 Nio: None zarr: 2.6.1 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.1.3 cartopy: None seaborn: 0.11.0 numbagg: None pint: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 3.2.1
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/4981/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue