home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 274298111

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
274298111 MDU6SXNzdWUyNzQyOTgxMTE= 1719 open_mfdataset crashes when files are present with datasets of dimension = 0 731499 closed 0     2 2017-11-15T20:44:17Z 2020-03-09T00:50:17Z 2020-03-09T00:50:17Z CONTRIBUTOR      

I have a bunch of netCDF files that I want to read through open_mfdataset. Each file was created with xarray through to_netcdf() and contains several variables with a single time dimension. Most files look like this:

``` netcdf CAL_LID_L2_05kmCLay-Standard-V4-10.2008-07-01T01-54-46ZD.hdf_extract { dimensions: time = 112 ; variables: float lat(time) ; lat:_FillValue = NaNf ; float lon(time) ; lon:_FillValue = NaNf ; float elev(time) ; elev:_FillValue = NaNf ; double daynight(time) ; daynight:_FillValue = NaN ; double surf(time) ; surf:_FillValue = NaN ;

// global attributes: :_NCProperties = "version=1|netcdflibversion=4.4.1|hdf5libversion=1.8.17" ; } ```

if one of the files is empty, i.e. the length of the 'time' dimension is zero:

``` netcdf CAL_LID_L2_05kmCLay-Standard-V4-10.2008-01-01T00-37-48ZD.hdf_extract { dimensions: time = UNLIMITED ; // (0 currently) variables: float lat(time) ; lat:_FillValue = NaNf ; float lon(time) ; lon:_FillValue = NaNf ; float elev(time) ; elev:_FillValue = NaNf ; double daynight(time) ; daynight:_FillValue = NaN ; double surf(time) ; surf:_FillValue = NaN ;

// global attributes: :_NCProperties = "version=1|netcdflibversion=4.4.1|hdf5libversion=1.8.17" ; } ```

then open_mfdataset crashes with

python File "./test_map_elev_month_gl2.py", line 22, in main data = xr.open_mfdataset(files, concat_dim='time', autoclose=True) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/backends/api.py", line 505, in open_mfdataset **kwargs) for p in paths] File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/backends/api.py", line 505, in <listcomp> **kwargs) for p in paths] File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/backends/api.py", line 301, in open_dataset return maybe_decode_store(store, lock) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/backends/api.py", line 243, in maybe_decode_store lock=lock) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/core/dataset.py", line 1094, in chunk for k, v in self.variables.items()]) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/core/dataset.py", line 1094, in <listcomp> for k, v in self.variables.items()]) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/core/dataset.py", line 1089, in maybe_chunk return var.chunk(chunks, name=name2, lock=lock) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/xarray/core/variable.py", line 540, in chunk data = da.from_array(data, chunks, name=name, lock=lock) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/dask/array/core.py", line 1798, in from_array chunks = normalize_chunks(chunks, x.shape) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/dask/array/core.py", line 1758, in normalize_chunks for s, c in zip(shape, chunks)), ()) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/dask/array/core.py", line 1758, in <genexpr> for s, c in zip(shape, chunks)), ()) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/dask/array/core.py", line 881, in blockdims_from_blockshape for d, bd in zip(shape, chunks)) File "/home/noel/.conda/envs/python3/lib/python3.6/site-packages/dask/array/core.py", line 881, in <genexpr> for d, bd in zip(shape, chunks)) ZeroDivisionError: integer division or modulo by zero

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1719/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 2 rows from issue in issue_comments
Powered by Datasette · Queries took 0.711ms · About: xarray-datasette