home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 539757754

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/3277#issuecomment-539757754 https://api.github.com/repos/pydata/xarray/issues/3277 539757754 MDEyOklzc3VlQ29tbWVudDUzOTc1Nzc1NA== 47371188 2019-10-09T00:18:34Z 2019-10-09T00:21:03Z NONE

Using xarray=0.13.0, the chunking behaviour has changed again (but still incorrect): ``` <xarray.DataArray (item: 300, day: 7653)> dask.array<xarray-\<this-array>, shape=(300, 7653), dtype=float64, chunksize=(20, 7653), chunktype=numpy.ndarray> Coordinates: * item (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299' * day (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01

<xarray.DataArray (item: 300, day: 7653)> dask.array<where, shape=(300, 7653), dtype=float64, chunksize=(20, 7648), chunktype=numpy.ndarray> Coordinates: * item (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299' * day (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01


ValueError
... ```

Note the chunksize is now (20, 7648) instead of (20, 7653).

I believe it might be related to https://github.com/pydata/xarray/pull/2942 -> I think disabling bottleneck for dask arrays for the rolling operation caused the bug above to appear (so the bug may have been there for a while, but doesn't appear because bottleneck was being used).

Doing a quick trace, in rolling.py , I think it's the line windows = self.construct(rolling_dim) in the reduce function which creates windows with incorrect chunking, possibly as a consequence of some transpose operations and dimension mix up?

It seems strange that other applications aren't having problems with this, unless I am doing something different in my code? Note that I am very specifically chunking in a different dimension to the rolling operation.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  488547784
Powered by Datasette · Queries took 0.72ms · About: xarray-datasette