issue_comments

Table actions

4 rows where author_association = "MEMBER" and issue = 1307112340 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • `interp` performance with chunked dimensions · 4 ✖

author_association 1

  • MEMBER · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1317516314 https://github.com/pydata/xarray/issues/6799#issuecomment-1317516314 https://api.github.com/repos/pydata/xarray/issues/6799 IC_kwDOAMm_X85Oh7Qa dcherian 2448579 2022-11-16T18:55:00Z 2022-11-16T18:55:00Z MEMBER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   `interp` performance with chunked dimensions 1307112340
1317358777 https://github.com/pydata/xarray/issues/6799#issuecomment-1317358777 https://api.github.com/repos/pydata/xarray/issues/6799 IC_kwDOAMm_X85OhUy5 dcherian 2448579 2022-11-16T17:04:23Z 2022-11-16T17:04:23Z MEMBER

The challenge is you could be interping to an unordered set of locations.

So perhaps we can sort the input locations, do the interp with map_overlap, then argsort the result back to expected order.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   `interp` performance with chunked dimensions 1307112340
1194294204 https://github.com/pydata/xarray/issues/6799#issuecomment-1194294204 https://api.github.com/repos/pydata/xarray/issues/6799 IC_kwDOAMm_X85HL3u8 dcherian 2448579 2022-07-25T16:07:09Z 2022-07-25T16:07:09Z MEMBER

The current code also has the unfortunate side-effect of merging all chunks too.

I think we should instead think of generating a dask array of weights and then using xr.dot

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   `interp` performance with chunked dimensions 1307112340
1187699270 https://github.com/pydata/xarray/issues/6799#issuecomment-1187699270 https://api.github.com/repos/pydata/xarray/issues/6799 IC_kwDOAMm_X85GytpG dcherian 2448579 2022-07-18T16:18:09Z 2022-07-18T16:18:21Z MEMBER

Given the performance behavior I'm guessing we may be doing sequntial interpolation for the dimensions, basically an interp1d call for all the xx points and from there another to the yy points, which for even a small number of points would require nearly all chunks to be loaded in.

Yeah I think this is right.

You could check if it was better before https://github.com/pydata/xarray/pull/4155 (if it worked that is)

cc @pums974 @Illviljan 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   `interp` performance with chunked dimensions 1307112340

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);