home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 1452521051

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/7021#issuecomment-1452521051 https://api.github.com/repos/pydata/xarray/issues/7021 1452521051 IC_kwDOAMm_X85Wk7Zb 39671216 2023-03-02T20:47:11Z 2023-03-02T20:47:11Z CONTRIBUTOR

Been half a year and I found myself stuck at this inconsistent behavior again. Another problem I found but haven't mentioned yet is that DatasetRolling.construct will swap the rolling dimension name with window_dim when DataArrayRolling.construct doesn't.

This time, I've actually identified a cause for this problem below:

https://github.com/pydata/xarray/blob/b018442c8dfa3e71ec35e294de69e2011949afec/xarray/core/rolling.py#L789-L791

python .isel({d: slice(None, None, s) for d, s in zip(self.dim, strides)})

I currently still can't figure it out what is the original intention that .isel trying to achieve since it causes so much problem without any benefit. It should be noted that this can explode the memory if xr.Dataset is reasonably large (It just explode 3 channels PPG, 135Hz, 6Hrs of recording, a mere 300MB to 20-40GB++, so I think this is critical).

Solution

Removing .isel part fixed everything.

Test case

python test_arr = xr.DataArray(np.arange(8).reshape(2, 4), dims=('a', 'b')) # Borrowed from `DataArray.__doc__`'s example. test_dset= xr.Dataset(data_vars={i: tr for i in range(3)}) DataArray ```python tr.rolling(b=2).construct('window_dim', stride=2)

<xarray.DataArray (a: 2, b: 2, window_dim: 2)> array([[[nan, 0.], [ 1., 2.]],

   [[nan,  4.],
    [ 5.,  6.]]])

Dimensions without coordinates: a, b, window_dim Datasetpython trd.rolling(b=2).construct('window_dim', stride=2)

<xarray.Dataset> Dimensions: (a: 2, b: 2, window_dim: 2) Dimensions without coordinates: a, b, window_dim Data variables: 0 (a, b, window_dim) float64 nan 0.0 1.0 2.0 nan 4.0 5.0 6.0 1 (a, b, window_dim) float64 nan 0.0 1.0 2.0 nan 4.0 5.0 6.0 2 (a, b, window_dim) float64 nan 0.0 1.0 2.0 nan 4.0 5.0 6.0

trd.rolling(b=2).construct('window_dim', stride=2)[0]

<xarray.DataArray 0 (a: 2, b: 2, window_dim: 2)> array([[[nan, 0.], [ 1., 2.]],

   [[nan,  4.],
    [ 5.,  6.]]])

Dimensions without coordinates: a, b, window_dim ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1370063272
Powered by Datasette · Queries took 0.673ms · About: xarray-datasette