home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 417066100

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2389#issuecomment-417066100 https://api.github.com/repos/pydata/xarray/issues/2389 417066100 MDEyOklzc3VlQ29tbWVudDQxNzA2NjEwMA== 1217238 2018-08-29T18:55:39Z 2018-08-29T18:55:39Z MEMBER

I don't really know how they work, but maybe pickeling those NetCDF4ArrayWrapper objects is expensive (ie they contain a reference to something they shouldn't)?

This seems plausible to me, though the situation is likely improved with #2261. It would be nice if dask had a way to consolidate the serialization of these objects, rather than separately serializing them in each task. It's not obvious to me how to do that in xarray short of manually building task graphs so those NetCDF4ArrayWrapper objects are created by dedicated tasks.

CC @mrocklin in case he has thoughts here

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  355264812
Powered by Datasette · Queries took 77.598ms · About: xarray-datasette