home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 564240510

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
564240510 MDU6SXNzdWU1NjQyNDA1MTA= 3767 ValueError when reading netCDF 16228337 closed 0     2 2020-02-12T20:08:45Z 2022-04-09T02:24:48Z 2022-04-09T02:24:48Z NONE      

MCVE Code Sample

```python

ds = xr.open_dataset('20090327_0600')

```

Problem Description

Whenever I try to read certain netCDF files it raises a ValueError: When changing to a larger dtype, its size must be a divisor of the total size in bytes of the last axis of the array. Here is a link to the one of the files that is raising the error: https://madis-data.ncep.noaa.gov/madisPublic1/data/archive/2009/03/27/LDAD/hfmetar/netCDF/20090327_0600.gz I don't have any issues reading this file though: https://madis-data.ncep.noaa.gov/madisPublic1/data/archive/2019/05/15/LDAD/hfmetar/netCDF/20190515_1200.gz

The traceback looks like this: ```python KeyError Traceback (most recent call last) ~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock) 197 try: --> 198 file = self._cache[self._key] 199 except KeyError:

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/lru_cache.py in getitem(self, key) 52 with self._lock: ---> 53 value = self._cache[key] 54 self._cache.move_to_end(key)

KeyError: [<function _open_scipy_netcdf at 0x11c8fc160>, ('/Users/jmiller/data/madis/20090327_0600',), 'r', (('mmap', None), ('version', 2))]

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) <ipython-input-26-04ef422e5840> in <module> ----> 1 ds = xr.open_dataset('madis/20090327_0600')

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, autoclose, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables, backend_kwargs, use_cftime) 536 537 with close_on_error(store): --> 538 ds = maybe_decode_store(store) 539 540 # Ensure source filename always stored in dataset object (GH issue #2550)

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/api.py in maybe_decode_store(store, lock) 444 445 def maybe_decode_store(store, lock=False): --> 446 ds = conventions.decode_cf( 447 store, 448 mask_and_scale=mask_and_scale,

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/conventions.py in decode_cf(obj, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables, use_cftime) 568 encoding = obj.encoding 569 elif isinstance(obj, AbstractDataStore): --> 570 vars, attrs = obj.load() 571 extra_coords = set() 572 file_obj = obj

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/common.py in load(self) 121 """ 122 variables = FrozenDict( --> 123 (_decode_variable_name(k), v) for k, v in self.get_variables().items() 124 ) 125 attributes = FrozenDict(self.get_attrs())

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/scipy_.py in get_variables(self) 155 def get_variables(self): 156 return FrozenDict( --> 157 (k, self.open_store_variable(k, v)) for k, v in self.ds.variables.items() 158 ) 159

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/scipy_.py in ds(self) 144 @property 145 def ds(self): --> 146 return self._manager.acquire() 147 148 def open_store_variable(self, name, var):

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/file_manager.py in acquire(self, needs_lock) 178 An open file object, as returned by opener(*args, **kwargs). 179 """ --> 180 file, _ = self._acquire_with_cache_info(needs_lock) 181 return file 182

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock) 202 kwargs = kwargs.copy() 203 kwargs["mode"] = self._mode --> 204 file = self._opener(self._args, *kwargs) 205 if self._mode == "w": 206 # ensure file doesn't get overriden when opened again

~/miniconda3/envs/proc/lib/python3.8/site-packages/xarray/backends/scipy_.py in _open_scipy_netcdf(filename, mode, mmap, version) 81 82 try: ---> 83 return scipy.io.netcdf_file(filename, mode=mode, mmap=mmap, version=version) 84 except TypeError as e: # netcdf3 message is obscure in this case 85 errmsg = e.args[0]

~/miniconda3/envs/proc/lib/python3.8/site-packages/scipy/io/netcdf.py in init(self, filename, mode, mmap, version, maskandscale) 282 283 if mode in 'ra': --> 284 self._read() 285 286 def setattr(self, attr, value):

~/miniconda3/envs/proc/lib/python3.8/site-packages/scipy/io/netcdf.py in _read(self) 614 self._read_dim_array() 615 self._read_gatt_array() --> 616 self._read_var_array() 617 618 def _read_numrecs(self):

~/miniconda3/envs/proc/lib/python3.8/site-packages/scipy/io/netcdf.py in _read_var_array(self) 720 # Build rec array. 721 if self.use_mmap: --> 722 rec_array = self._mm_buf[begin:begin+self._recs*self._recsize].view(dtype=dtypes) 723 rec_array.shape = (self._recs,) 724 else:

ValueError: When changing to a larger dtype, its size must be a divisor of the total size in bytes of the last axis of the array. ```

Output of xr.show_versions()

# Paste the output here xr.show_versions() here INSTALLED VERSIONS ------------------ commit: None python: 3.8.1 | packaged by conda-forge | (default, Jan 29 2020, 15:06:10) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.2.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.0 pandas: 1.0.0 numpy: 1.18.1 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.3 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 45.1.0.post20200119 pip: 20.0.2 conda: None pytest: None IPython: 7.12.0 sphinx: None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3767/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 2 rows from issue in issue_comments
Powered by Datasette · Queries took 2.478ms · About: xarray-datasette