issue_comments

Table actions

5 rows where issue = 305702311 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

issue 1

  • DataArray.rolling().mean() is way slower than it should be · 5 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
373552913 https://github.com/pydata/xarray/issues/1993#issuecomment-373552913 https://api.github.com/repos/pydata/xarray/issues/1993 MDEyOklzc3VlQ29tbWVudDM3MzU1MjkxMw== fujiisoup 6815844 2018-03-15T23:17:37Z 2018-03-15T23:17:37Z MEMBER

I had to improve this in #1837, but I did not notice that just creating slices takes so long >_< Sent a PR.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   DataArray.rolling().mean() is way slower than it should be 305702311
373540862 https://github.com/pydata/xarray/issues/1993#issuecomment-373540862 https://api.github.com/repos/pydata/xarray/issues/1993 MDEyOklzc3VlQ29tbWVudDM3MzU0MDg2Mg== max-sixty 5635139 2018-03-15T22:18:52Z 2018-03-15T22:45:24Z MEMBER

(I don't know who wrote this code but woof!)

🙈

Edit: Whoever wrote that code is a hero for contributing so much code - the likeliest sources of bad code are the most prolific and valuable contributors

 
{
    "total_count": 3,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 3,
    "rocket": 0,
    "eyes": 0
}
   DataArray.rolling().mean() is way slower than it should be 305702311
373540541 https://github.com/pydata/xarray/issues/1993#issuecomment-373540541 https://api.github.com/repos/pydata/xarray/issues/1993 MDEyOklzc3VlQ29tbWVudDM3MzU0MDU0MQ== max-sixty 5635139 2018-03-15T22:17:27Z 2018-03-15T22:17:27Z MEMBER

I don't think we use anything created in _setup_windows() until we call __iter__, which IIUC is only called on non-vectorized methods.

We could: - Combine _setup_windows and __iter__: only runs the code when you need it, but would run each time __iter__ was called - Combine _setup_windows and __iter__ and cache the result: more code but only need to run the setup code once - Convert it to a generator: lower up-front cost, but more engineering and doesn't make it cheaper assuming you consumed the whole iterator anyway

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   DataArray.rolling().mean() is way slower than it should be 305702311
373540063 https://github.com/pydata/xarray/issues/1993#issuecomment-373540063 https://api.github.com/repos/pydata/xarray/issues/1993 MDEyOklzc3VlQ29tbWVudDM3MzU0MDA2Mw== jhamman 2443309 2018-03-15T22:15:14Z 2018-03-15T22:15:14Z MEMBER

+1 from me on making the window creation lazy. (I don't know who wrote this code but woof!)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   DataArray.rolling().mean() is way slower than it should be 305702311
373538852 https://github.com/pydata/xarray/issues/1993#issuecomment-373538852 https://api.github.com/repos/pydata/xarray/issues/1993 MDEyOklzc3VlQ29tbWVudDM3MzUzODg1Mg== max-sixty 5635139 2018-03-15T22:09:47Z 2018-03-15T22:09:47Z MEMBER

Quick discovery - it looks like it's spending all the time creating 9m slice objects here: python self.window_indices = [slice(start, stop) for start, stop in zip(starts, stops)]

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   DataArray.rolling().mean() is way slower than it should be 305702311

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);