issue_comments

Table actions

2 rows where issue = 1465047346 and user = 1217238 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • shoyer · 2 ✖

issue 1

  • (Issue #7324) added functions that return data values in memory efficient manner · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1328156723 https://github.com/pydata/xarray/pull/7323#issuecomment-1328156723 https://api.github.com/repos/pydata/xarray/issues/7323 IC_kwDOAMm_X85PKhAz shoyer 1217238 2022-11-27T02:31:51Z 2022-11-27T02:31:51Z MEMBER

Use cases would be in any web service that would like to provide the final data values back to a user in JSON.

For what it's worth, I think your users will have a poor experience with encoded JSON data for very large arrays. It will be slow to compress and transfer this data.

In the long term, you would probably do better to transmit the data in some binary form (e.g., by calling tobytes() on the underlying np.ndarray objects, or by using Xarray's to_netcdf).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   (Issue #7324) added functions that return data values in memory efficient manner 1465047346
1328156304 https://github.com/pydata/xarray/pull/7323#issuecomment-1328156304 https://api.github.com/repos/pydata/xarray/issues/7323 IC_kwDOAMm_X85PKg6Q shoyer 1217238 2022-11-27T02:27:07Z 2022-11-27T02:27:07Z MEMBER

Thanks for report and the PR!

This really needs a "minimal complete verifiable" example (e.g., by creating and loading a Zarr array with random data) so others can verify your reported the performance gains: https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports https://stackoverflow.com/help/minimal-reproducible-example

To be honest, this fix looks a little funny to me, because NumPy's own implementation of tolist() is so similar. I would love to understand what is going on.

If you can reproduce the issue only using NumPy, it could also make more sense to file this as a upstream bug report to NumPy. The NumPy maintainers are in a better position to debug tricky memory allocation issues involving NumPy.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   (Issue #7324) added functions that return data values in memory efficient manner 1465047346

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);