issue_comments

Table actions

7 rows where issue = 429511994 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

author_association 2

issue 1

  • Very slow coordinate assignment with dask array · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
567585193 https://github.com/pydata/xarray/issues/2867#issuecomment-567585193 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDU2NzU4NTE5Mw== dcherian 2448579 2019-12-19T17:28:09Z 2019-12-19T17:28:09Z MEMBER

Great! thanks for checking.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
567583795 https://github.com/pydata/xarray/issues/2867#issuecomment-567583795 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDU2NzU4Mzc5NQ== jbusecke 14314623 2019-12-19T17:24:27Z 2019-12-19T17:24:27Z CONTRIBUTOR

I can confirm that this issue is resolved for my project. Seems to not make a difference in speed anymore whether I assign the dataarray as coordinate or data variable. Thanks for the fix!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
566689340 https://github.com/pydata/xarray/issues/2867#issuecomment-566689340 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDU2NjY4OTM0MA== jbusecke 14314623 2019-12-17T18:28:32Z 2019-12-17T18:28:32Z CONTRIBUTOR

I think this issue was actually a dupe. I remember you pointing me to changes in 14.x, that improved the performance, but I cant find the other issue right now. I will have an opportunity to test this in the coming days on some huge GFDL data

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
566637811 https://github.com/pydata/xarray/issues/2867#issuecomment-566637811 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDU2NjYzNzgxMQ== dcherian 2448579 2019-12-17T16:23:23Z 2019-12-17T16:23:23Z MEMBER

It may be. It would be good to check if you have the time.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
566636980 https://github.com/pydata/xarray/issues/2867#issuecomment-566636980 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDU2NjYzNjk4MA== jbusecke 14314623 2019-12-17T16:21:23Z 2019-12-17T16:21:23Z CONTRIBUTOR

I believe this was fixed in a recent version. Closing

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
481417930 https://github.com/pydata/xarray/issues/2867#issuecomment-481417930 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDQ4MTQxNzkzMA== jbusecke 14314623 2019-04-09T20:16:18Z 2019-04-09T20:16:18Z CONTRIBUTOR

Could you think of a way I would be able to diagnose this further? Sorry for these wide questions but I am not very familiar with these xarray internals.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994
480640557 https://github.com/pydata/xarray/issues/2867#issuecomment-480640557 https://api.github.com/repos/pydata/xarray/issues/2867 MDEyOklzc3VlQ29tbWVudDQ4MDY0MDU1Nw== shoyer 1217238 2019-04-07T23:17:43Z 2019-04-07T23:17:43Z MEMBER

But I suspect that something triggers the computation of the array, when assigning a coordinate.

This sounds correct to me.

I'm a little surprised about a significant performance between these two ways of adding coordinates. The implementation of these methods does differ slightly, but they should be pretty similar.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Very slow coordinate assignment with dask array 429511994

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);