home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 372856076

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/1823#issuecomment-372856076 https://api.github.com/repos/pydata/xarray/issues/1823 372856076 MDEyOklzc3VlQ29tbWVudDM3Mjg1NjA3Ng== 14314623 2018-03-13T23:40:54Z 2018-03-13T23:40:54Z CONTRIBUTOR

Would these two options be necessarily mutually exclusive?

I think parallelizing the read in sounds amazing.

But isnt there some merit in skipping some of the checks all together, if the user is sure about the structure of the data contained in the many files?

I am often working with the aforementioned type of data (many files either contain a new timestep or a different variable, but most of the dimensions/coordinates are the same).

In some cases I am finding that reading the data "lazily" consumes a significant amount of the time in my workflow. I am unsure how hard this would be to achieve, and perhaps it is not worth it after all.

Just putting out a few ideas, while I wait for my xr.open_mfdataset to finish :-)

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 1,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  288184220
Powered by Datasette · Queries took 0.821ms · About: xarray-datasette