issues: 604210297
This data as json
| id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 604210297 | MDU6SXNzdWU2MDQyMTAyOTc= | 3991 | Nondeterministic bug with bytestring decoding | 23484003 | closed | 0 | 2 | 2020-04-21T18:57:28Z | 2020-04-21T20:39:56Z | 2020-04-21T20:39:56Z | NONE | I have an HDF5 dataset with a scalar variable called 'name' that is actual a 0-D NumPy array with dtype '|S8'. (Not my choice, this is what I get from someone else...) Occasionally, the loading fails. MCVE Code Sample```python Set up the fileimport h5py f = h5py.File("error_demo.h5",mode='w') f.create_dataset('name',shape=(),dtype="|S8",data=np.array([b'f(Pt,TE)'],dtype='|S8')) f.close() Produce the error -- you may need to adjust the number of times you run the loopimport xarray as xr for i in range(10): xr.load_dataset("error_demo.h5") ``` Expected Output<xarray.Dataset> Dimensions: () Data variables: name <U8 'f(Pt,TE)' Problem DescriptionThe resulting error message``` Traceback (most recent call last): File "<ipython-input-3-b8e48f28a262>", line 1, in <module> mcout62 = xr.load_dataset("57062/mcout000011.h5",group=r"part/ions/dE(r,z,D)") File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/backends/api.py", line 261, in load_dataset return ds.load() File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/dataset.py", line 659, in load v.load() File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/variable.py", line 375, in load self._data = np.asarray(self._data) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray return array(a, dtype, copy=False, order=order) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 677, in array self._ensure_cached() File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 674, in _ensure_cached self.array = NumpyIndexingAdapter(np.asarray(self.array)) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray return array(a, dtype, copy=False, order=order) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 653, in array return np.asarray(self.array, dtype=dtype) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray return array(a, dtype, copy=False, order=order) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 557, in array return np.asarray(array[self.key], dtype=None) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 73, in getitem key, self.shape, indexing.IndexingSupport.OUTER, self._getitem File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/core/indexing.py", line 837, in explicit_indexing_adapter result = raw_indexing_method(raw_key.tuple) File "/Users/lmorton/opt/anaconda3/lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 85, in _getitem array = getitem(original_array, key) File "netCDF4/_netCDF4.pyx", line 4408, in netCDF4._netCDF4.Variable.getitem File "netCDF4/_netCDF4.pyx", line 5384, in netCDF4._netCDF4.Variable._get UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 9: invalid start byte ``` VersionsOutput of <tt>xr.show_versions()</tt>INSTALLED VERSIONS ------------------ commit: None python: 3.7.6 (default, Jan 8 2020, 13:42:34) [Clang 4.0.1 (tags/RELEASE_401/final)] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.7.3 xarray: 0.15.0 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.11.0 distributed: 2.11.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.0.0.post20200309 pip: 20.0.2 conda: 4.8.3 pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.0 |
{
"url": "https://api.github.com/repos/pydata/xarray/issues/3991/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
} |
completed | 13221727 | issue |