issue_comments

Table actions

7 rows where issue = 712052219 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

author_association 2

issue 1

  • Implement rolling_exp for dask arrays · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
702363119 https://github.com/pydata/xarray/issues/4474#issuecomment-702363119 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjM2MzExOQ== max-sixty 5635139 2020-10-01T19:55:11Z 2020-10-01T19:55:11Z MEMBER

Thanks @heerad . Looks like we're doing it with overlaps rather than any rolling-specific algo.

Agree that approach won't work with exp. OTOH your suggestion could be generalized to improve performance in all rolling operations, depending on the relative costs of sending the underlying overlapping data vs. sending only the edges of the numerator / denominator and doing another pass.

If anyone has experience with similar operations, feel free to offer suggestions.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
702346076 https://github.com/pydata/xarray/issues/4474#issuecomment-702346076 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjM0NjA3Ng== heerad 2560426 2020-10-01T19:20:50Z 2020-10-01T19:23:31Z NONE 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
702340842 https://github.com/pydata/xarray/issues/4474#issuecomment-702340842 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjM0MDg0Mg== max-sixty 5635139 2020-10-01T19:10:24Z 2020-10-01T19:10:42Z MEMBER

OK, I don't see references there to dask, but thanks for letting me know. Do you know where the code for rolling over chunks is?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
702331156 https://github.com/pydata/xarray/issues/4474#issuecomment-702331156 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjMzMTE1Ng== heerad 2560426 2020-10-01T18:52:18Z 2020-10-01T18:52:18Z NONE

Yes, see http://xarray.pydata.org/en/stable/computation.html#rolling-window-operations.

rolling works with dask, but rolling_exp does not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
702327698 https://github.com/pydata/xarray/issues/4474#issuecomment-702327698 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjMyNzY5OA== max-sixty 5635139 2020-10-01T18:45:40Z 2020-10-01T18:45:40Z MEMBER

I have less experience with dask than others — are simple rolling means already implemented across chunks? That would be a much easier place to start...

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
702181324 https://github.com/pydata/xarray/issues/4474#issuecomment-702181324 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMjE4MTMyNA== heerad 2560426 2020-10-01T14:39:01Z 2020-10-01T14:39:01Z NONE

Great! This will be a common use-case for me, and I imagine others who are doing any sort of time series computation on large datasets.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219
701648525 https://github.com/pydata/xarray/issues/4474#issuecomment-701648525 https://api.github.com/repos/pydata/xarray/issues/4474 MDEyOklzc3VlQ29tbWVudDcwMTY0ODUyNQ== max-sixty 5635139 2020-09-30T21:11:30Z 2020-09-30T21:11:30Z MEMBER

I think we'd be open to this

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Implement rolling_exp for dask arrays 712052219

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);