home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 306009664

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/1440#issuecomment-306009664 https://api.github.com/repos/pydata/xarray/issues/1440 306009664 MDEyOklzc3VlQ29tbWVudDMwNjAwOTY2NA== 1217238 2017-06-04T00:28:19Z 2017-06-04T00:28:19Z MEMBER

My main concern is that netCDF4 chunk sizes (e.g., ~10-100KB in that blog post) are often much smaller than well sized dask chunks (10-100MB, per the Dask FAQ).

I do think it would be appropriate to issue a warning if you are making dask chunks that don't line up nicely with chunks on disk to avoid performance issues (in general each chunk on disk should usually end up on only one chunk in dask), but there are lots of options for aggregating to larger chunks and it's hard to choose the best way to do that without knowing how the data will be used.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  233350060
Powered by Datasette · Queries took 1.57ms · About: xarray-datasette