home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 662868741

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/4236#issuecomment-662868741 https://api.github.com/repos/pydata/xarray/issues/4236 662868741 MDEyOklzc3VlQ29tbWVudDY2Mjg2ODc0MQ== 8098361 2020-07-23T07:53:23Z 2020-07-23T07:53:23Z NONE

My minimal functools.partial has some weird behaviour. ``` import xarray as xr from functools import partial from pathlib import Path

def preprocessing(doys, ds):

print(doys)

ds = ds.sel(time=((ds['time.dayofyear'] >= doys[0])
            & (ds['time.dayofyear'] < doys[1])))
return ds

def get_data_set(doys, parallel=True): ds = xr.open_mfdataset( files, combine='nested', concat_dim='time', parallel=parallel, preprocess=partial(preprocessing, doys) ) return ds

if name == 'main': pth = "/path/to/data" day_of_year_range = (100, 140) files = list(Path(pth).rglob('*.nc')) ds = get_data_set(day_of_year_range, parallel=False) print(ds) If I run with `parallel=True` the python kernel crashes, or I get something like File "netCDF4_netCDF4.pyx", line 2344, in netCDF4._netCDF4.Dataset.init File "netCDF4_netCDF4.pyx", line 1789, in netCDF4._netCDF4._get_vars File "netCDF4_netCDF4.pyx", line 1887, in netCDF4._netCDF4._ensure_nc_success RuntimeError: NetCDF: Can't open HDF5 attribute `` Ifparallel=False` (same set of input files) everything is OK. Passing a new day of year range works, it's all good.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  659142789
Powered by Datasette · Queries took 0.634ms · About: xarray-datasette