home / github / commits

Menu
  • GraphQL API
  • Search all tables

commits: cbeb70d99f391b015724fa75411e6860ed097a54

This data as json

sha message author_date committer_date raw_author raw_committer repo author committer
cbeb70d99f391b015724fa75411e6860ed097a54 Use deterministic names for dask arrays from open_dataset This will allow xray users to take advantage of dask's nascent support for caching intermediate results (https://github.com/blaze/dask/pull/502). For example: In [1]: import xray In [2]: from dask.diagnostics.cache import Cache In [3]: c = Cache(5e7) In [4]: c.register() In [5]: ds = xray.open_mfdataset('/Users/shoyer/data/era-interim/2t/2014-*.nc', engine='scipy') In [6]: %time ds.sum().load() CPU times: user 2.72 s, sys: 2.7 s, total: 5.41 s Wall time: 3.85 s Out[6]: <xray.Dataset> Dimensions: () Coordinates: *empty* Data variables: t2m float64 5.338e+10 In [7]: %time ds.mean().load() CPU times: user 5.31 s, sys: 1.86 s, total: 7.17 s Wall time: 1.81 s Out[7]: <xray.Dataset> Dimensions: () Coordinates: *empty* Data variables: t2m float64 279.0 In [8]: %time ds.mean().load() CPU times: user 7.73 ms, sys: 2.73 ms, total: 10.5 ms Wall time: 8.45 ms Out[8]: <xray.Dataset> Dimensions: () Coordinates: *empty* Data variables: t2m float64 279.0 2015-08-30T14:53:06Z 2015-09-01T20:41:21Z 7ac163783245dbec82e06aac57b56992d3cf2b72 7ac163783245dbec82e06aac57b56992d3cf2b72 13221727 1217238 1217238
Powered by Datasette · Queries took 0.688ms · About: xarray-datasette