home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

8 rows where issue = 931016490 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

  • Illviljan 4
  • max-sixty 3
  • github-actions[bot] 1

author_association 2

  • MEMBER 7
  • CONTRIBUTOR 1

issue 1

  • Do not transpose 1d arrays during interpolation · 8 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
886450425 https://github.com/pydata/xarray/pull/5542#issuecomment-886450425 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X8401ij5 Illviljan 14371165 2021-07-26T07:28:58Z 2021-07-26T07:33:57Z MEMBER

It probably is decently fast but it can't compare to just not running the .copy() code if a copy isn't necessary. :) But this isn't a major bottleneck either:

Before, the transpose is one of the largest bottlenecks in missing.interp where the .copy() is what takes time. missing.interp took 1.33s to run.

After, the transpose obviously isn't a factor anymore since it isn't triggered. missing.interp took 1.24s to run.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
886422293 https://github.com/pydata/xarray/pull/5542#issuecomment-886422293 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X8401bsV max-sixty 5635139 2021-07-26T06:41:38Z 2021-07-26T06:41:38Z MEMBER

It's curious that's slow — it's not a deep copy and so should be fast (in python terms!), since it's just copying the class instance.

Totally understand re ASV — and more generally you should choose the most meaningful work for you. I hope you continue to become more involved with the project, and there'll be plenty of time to expand into other areas.

(though down a level, we should have some way of justifying the merge — let me know if you have any profiles to hand, no rush)

Thanks as ever @Illviljan !

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
886261206 https://github.com/pydata/xarray/pull/5542#issuecomment-886261206 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X84000XW Illviljan 14371165 2021-07-25T21:38:49Z 2021-07-25T21:38:49Z MEMBER

Edit: actually is this already implemented? https://github.com/pydata/xarray/blob/main/xarray/core/variable.py#L1441-L1444. Does interpolate not hit this code path?

Yes, that's the copy that's slow and there's no real need to create a new copy in the 1D case. My (bad) idea was to just return the original array there instead, but as you noted that might not be fully intuitive.

A ASV for 1D- and ND-interpolation and having it running in the CI would be nice. Though I'm not really interested in implementing that, It would be just another thing to document for me because I use other profilers anyway and as you said getting it setup requires some work that I don't enjoy.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
886253326 https://github.com/pydata/xarray/pull/5542#issuecomment-886253326 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X8400ycO max-sixty 5635139 2021-07-25T20:25:15Z 2021-07-25T21:19:01Z MEMBER

~Yeah, I guess if someone does x = y.transpose() and then x[0] = 42, then y would be inconsistently updated. Don't think we're likely to get copy-on-write semantics soon!~

~So maybe this is the best we can hope for.~

Edit: actually is this already implemented? https://github.com/pydata/xarray/blob/main/xarray/core/variable.py#L1441-L1444. Does interpolate not hit this code path?

Is it worth adding an ASV? I've found them fairly quick to set up a new one, though takes some lift to set up the environment etc. I think in general we should try and have them for performance work, so we can track if it regresses.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
886247722 https://github.com/pydata/xarray/pull/5542#issuecomment-886247722 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X8400xEq Illviljan 14371165 2021-07-25T19:36:12Z 2021-07-25T19:36:12Z MEMBER

Yes this improves the interp performance a bit. The .copy in the transpose is rather slow so it seems better to just not do it. An alternative is removing the .copy in the transpose or make it an option?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
886242665 https://github.com/pydata/xarray/pull/5542#issuecomment-886242665 https://api.github.com/repos/pydata/xarray/issues/5542 IC_kwDOAMm_X8400v1p max-sixty 5635139 2021-07-25T18:50:46Z 2021-07-25T18:50:46Z MEMBER

Sorry this didn't get @Illviljan .

It looks good, without me having that much context. Do you have any info on whether this has any performance impact?

I imagine this isn't as easy as it sounds, but do you have a view on whether we could apply this concept more broadly, and make transposing 1D arrays a no-op in the transpose method, rather than writing that logic for each method that calls .transpose?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
869225736 https://github.com/pydata/xarray/pull/5542#issuecomment-869225736 https://api.github.com/repos/pydata/xarray/issues/5542 MDEyOklzc3VlQ29tbWVudDg2OTIyNTczNg== github-actions[bot] 41898282 2021-06-27T21:24:07Z 2021-07-22T08:46:01Z CONTRIBUTOR

Unit Test Results

6 files           6 suites   52m 1s :stopwatch: 16 204 tests 14 485 :heavy_check_mark: 1 719 :zzz: 0 :x: 90 420 runs  82 260 :heavy_check_mark: 8 160 :zzz: 0 :x:

Results for commit fb68d59b.

:recycle: This comment has been updated with latest results.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490
869223054 https://github.com/pydata/xarray/pull/5542#issuecomment-869223054 https://api.github.com/repos/pydata/xarray/issues/5542 MDEyOklzc3VlQ29tbWVudDg2OTIyMzA1NA== Illviljan 14371165 2021-06-27T21:00:55Z 2021-06-27T21:01:17Z MEMBER

It's just a simple copy/paste job at the moment. I welcome suggestions for a more elegant solution.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Do not transpose 1d arrays during interpolation 931016490

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 15.056ms · About: xarray-datasette