home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 358017392

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/1832#issuecomment-358017392 https://api.github.com/repos/pydata/xarray/issues/1832 358017392 MDEyOklzc3VlQ29tbWVudDM1ODAxNzM5Mg== 1197350 2018-01-16T16:21:37Z 2018-01-16T16:21:37Z MEMBER

Below is how I work around the issue in practice: writing a loop over each item in the groupby, and then looping over each variable, loading, and writing to disk.

python gb = ds.groupby('time.month') for month, dsm in gb: dsm_anom2 = ((dsm - ds_mm.sel(month=month))**2).mean(dim='time') dsm_anom2 = dsm_anom2.rename({f: f + '2' for f in fields}) dsm_anom2.coords['month'] = month for var in dsm_anom2.data_vars: filename = save_dir + '%02d.%s_%s.nc' % (month, prefix, var) print(filename) ds_out = dsm_anom2[[var]].load() ds_out.to_netcdf(filename)

Needless to say, this feels more like my pre-xarray/dask workflow.

Since @mrocklin has made it pretty clear that dask will not automatically solve this for us any time soon, we need to brainstorm some creative ways to make this extremely common use case more friendly with out-of-core data.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  288785270
Powered by Datasette · Queries took 0.544ms · About: xarray-datasette