issues: 289556132

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
289556132 MDExOlB1bGxSZXF1ZXN0MTYzNjU3NDI0 1837 Rolling window with `as_strided` 6815844 closed 0     14 2018-01-18T09:18:19Z 2018-06-22T22:27:11Z 2018-03-01T03:39:19Z MEMBER   0 pydata/xarray/pulls/1837
  • [x] Closes #1831, #1142, #819
  • [x] Tests added
  • [x] Tests passed
  • [x] Passes git diff upstream/master **/*py | flake8 --diff
  • [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I started to work for refactoring rollings. As suggested in #1831 comment, I implemented rolling_window methods based on as_strided.

I got more than 1,000 times speed up! yey!

python In [1]: import numpy as np ...: import xarray as xr ...: ...: da = xr.DataArray(np.random.randn(10000, 3), dims=['x', 'y']) with the master python %timeit da.rolling(x=5).reduce(np.mean) 1 loop, best of 3: 9.68 s per loop with the current implementation python %timeit da.rolling(x=5).reduce(np.mean) 100 loops, best of 3: 5.29 ms per loop and with the bottleneck python %timeit da.rolling(x=5).mean() 100 loops, best of 3: 2.62 ms per loop

My current concerns are + Can we expose the new rolling_window method of DataArray and Dataset to the public? I think this method itself is useful for many usecases, such as short-term-FFT and convolution. This also gives more flexible rolling operation, such as windowed moving average, strided rolling, and ND-rolling.

  • Is there any dask's equivalence to numpy's as_strided? Currently, I just use a slice->concatenate path, but I don't think it is very efficient. (Is it already efficient, as dask utilizes out-of-core computation?)

Any thoughts are welcome.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1837/reactions",
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     13221727 pull

Links from other tables