issue_comments: 439281383

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2554#issuecomment-439281383 https://api.github.com/repos/pydata/xarray/issues/2554 439281383 MDEyOklzc3VlQ29tbWVudDQzOTI4MTM4Mw== 40218891 2018-11-16T04:50:43Z 2018-11-16T04:50:43Z NONE

The error RuntimeError: NetCDF: Bad chunk sizes. is unrelated to the original problem with segv crashes. It is caused by a bug in netcdf4 C library. It is fixed in the latest version 4.6.1. As of yesterday, the newest netcdf4-python manylinux wheel contains an older version. The solution is to build netcdf4-python from source.

The segv crashes occur with other datasets as well. Example test set I used:

for year in range(2000, 2005): file = '/tmp/dx{:d}.nc'.format(year) #times = pd.date_range('{:d}-01-01'.format(year), '{:d}-12-31'.format(year), name='time') times = pd.RangeIndex(year, year+300, name='time') v = np.array([np.random.random((32, 32)) for i in range(times.size)]) dx = xr.Dataset({'v': (('time', 'y', 'x'), v)}, {'time': times}) dx.to_netcdf(file, format='NETCDF4', encoding={'time': {'chunksizes': (1024,)}}, unlimited_dims='time')

A simple fix is to change the scheduler as I did in my original post.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   379472634