issue_comments

Table actions

5 rows where issue = 440233667 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

author_association 2

issue 1

  • test_rolling_wrapped_dask is failing with dask-master · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
506208573 https://github.com/pydata/xarray/issues/2940#issuecomment-506208573 https://api.github.com/repos/pydata/xarray/issues/2940 MDEyOklzc3VlQ29tbWVudDUwNjIwODU3Mw== shoyer 1217238 2019-06-27T06:20:19Z 2019-06-27T06:20:19Z MEMBER

https://github.com/pydata/xarray/pull/3040 is my preferred fix for the rolling window issue. I plan to merge it sometime tomorrow unless I get any comments....

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_rolling_wrapped_dask is failing with dask-master 440233667
504061610 https://github.com/pydata/xarray/issues/2940#issuecomment-504061610 https://api.github.com/repos/pydata/xarray/issues/2940 MDEyOklzc3VlQ29tbWVudDUwNDA2MTYxMA== zbarry 4762711 2019-06-20T15:03:22Z 2019-06-20T15:03:22Z NONE

@shoyer - yeah, I'd tend to agree. I had to do a bit of digging when I encountered this problem, because I was getting serialization errors (since it was doing away with the chunks on massive arrays) that I then eventually connected to this rolling problem. An explicit message that says "can't do this yet" would be good in light of this kind of situation.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_rolling_wrapped_dask is failing with dask-master 440233667
504058279 https://github.com/pydata/xarray/issues/2940#issuecomment-504058279 https://api.github.com/repos/pydata/xarray/issues/2940 MDEyOklzc3VlQ29tbWVudDUwNDA1ODI3OQ== shoyer 1217238 2019-06-20T14:55:04Z 2019-06-20T14:55:04Z MEMBER

Looking into this in more detail, the fix looks somewhat non-trivial. For example, we definitely are not padding by enough inside dask_rolling_wrapper if center=False: https://github.com/pydata/xarray/blob/145f25f69078f245313bb8e07b7c6af7509d0de8/xarray/core/dask_array_ops.py#L29

I'm not sure if this was ever tested properly, at least for dask arrays with multiple chunks. My guess is that this previously worked by consolidating dask arrays into a single chunk, which would simply fail for arrays that are larger than fit into memory.

For now, I think it would be safest to issue a new release that simply disables rolling() methods on dask arrays (by raising an error), and to save a full fix for later. I am concerned about letting the current behavior stick around, which silently calculates the wrong result.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_rolling_wrapped_dask is failing with dask-master 440233667
489961494 https://github.com/pydata/xarray/issues/2940#issuecomment-489961494 https://api.github.com/repos/pydata/xarray/issues/2940 MDEyOklzc3VlQ29tbWVudDQ4OTk2MTQ5NA== fujiisoup 6815844 2019-05-07T07:10:02Z 2019-05-07T07:10:02Z MEMBER

Sent a fix

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_rolling_wrapped_dask is failing with dask-master 440233667
489544700 https://github.com/pydata/xarray/issues/2940#issuecomment-489544700 https://api.github.com/repos/pydata/xarray/issues/2940 MDEyOklzc3VlQ29tbWVudDQ4OTU0NDcwMA== fujiisoup 6815844 2019-05-06T08:24:52Z 2019-05-06T08:24:52Z MEMBER

@shoyer

I have no idea yet... I will look inside later today.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_rolling_wrapped_dask is failing with dask-master 440233667

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);