home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 212561278

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
212561278 MDU6SXNzdWUyMTI1NjEyNzg= 1301 open_mfdataset() significantly slower on 0.9.1 vs. 0.8.2 10554254 closed 0     17 2017-03-07T21:16:53Z 2017-11-16T15:02:48Z 2017-11-16T15:02:00Z NONE      

I noticed a big speed discrepancy between xarray versions 0.8.2 and 0.9.1 when using open_mfdataset() on a dataset ~ 1.2 GB in size, consisting of 3 files and using netcdf4 as the engine. 0.8.2 was run first, so this is probably not a disk caching issue.

Test ``` import xarray as xr import time

start_time = time.time() ds0 = xr.open_mfdataset('./*.nc') print("--- %s seconds ---" % (time.time() - start_time)) ```

Result

xarray==0.8.2, dask==0.11.1, netcdf4==1.2.4 --- 0.736030101776 seconds --- xarray==0.9.1, dask==0.13.0, netcdf4==1.2.4

--- 52.2800869942 seconds ---

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1301/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 17 rows from issue in issue_comments
Powered by Datasette · Queries took 0.538ms · About: xarray-datasette