issues: 710876876

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
710876876 MDU6SXNzdWU3MTA4NzY4NzY= 4471 Numeric scalar variable attributes (including fill_value, scale_factor, add_offset) are 1-d instead of 0-d with h5netcdf engine, triggering ValueError: non-broadcastable output on application when loading single elements 500246 closed 0     13 2020-09-29T08:15:48Z 2020-10-11T20:06:33Z 2020-10-11T20:06:33Z CONTRIBUTOR      

What happened:

When I try to open a NetCDF file using the h5netcdf engine, accessing a single data point before scale factors have been applied results in ValueError: non-broadcastable output operand with shape () doesn't match the broadcast shape (1,). The MCVE (see below) results in:

Traceback (most recent call last): File "mwe93.py", line 4, in <module> ds["Rad"][400, 300].load() File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/dataarray.py", line 808, in load ds = self._to_temp_dataset().load(**kwargs) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/dataset.py", line 662, in load v.load() File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/variable.py", line 439, in load self._data = np.asarray(self._data) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray return array(a, dtype, copy=False, order=order) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/indexing.py", line 685, in __array__ self._ensure_cached() File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/indexing.py", line 682, in _ensure_cached self.array = NumpyIndexingAdapter(np.asarray(self.array)) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray return array(a, dtype, copy=False, order=order) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/indexing.py", line 655, in __array__ return np.asarray(self.array, dtype=dtype) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray return array(a, dtype, copy=False, order=order) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/core/indexing.py", line 560, in __array__ return np.asarray(array[self.key], dtype=None) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/numpy/core/_asarray.py", line 83, in asarray return array(a, dtype, copy=False, order=order) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/coding/variables.py", line 70, in __array__ return self.func(self.array) File "/data/gholl/miniconda3/envs/py38b/lib/python3.8/site-packages/xarray/coding/variables.py", line 220, in _scale_offset_decoding data *= scale_factor ValueError: non-broadcastable output operand with shape () doesn't match the broadcast shape (1,)

What you expected to happen:

I expect the data access to work similarly as when opening with other engines.

Minimal Complete Verifiable Example:

python import xarray fn = "/data/gholl/cache/fogtools/abi/2017/03/14/20/06/7/OR_ABI-L1b-RadF-M3C07_G16_s20170732006100_e20170732016478_c20170732016514.nc" with xarray.open_dataset(fn, engine="h5netcdf") as ds: ds["Rad"][400, 300].load()

Anything else we need to know?:

An earlier version of this issue, and some comments, refer to fsspec or working on open files, but that proved to have nothing to do with the problem.

Environment:

I've confirmed this issue installing xarray from latest master, which means xarray 0.16.2.dev11+gf821fe20 at the time of writing,

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.8.5 | packaged by conda-forge | (default, Sep 24 2020, 16:55:52) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.12.14-lp150.12.82-default machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.2.dev11+gf821fe20 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.6 cfgrib: None iris: None bottleneck: None dask: 2.27.0 distributed: 2.27.0 matplotlib: 3.3.2 cartopy: None seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200917 pip: 20.2.3 conda: None pytest: 6.0.2 IPython: None sphinx: None 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4471/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables