issue_comments

Table actions

1 row where issue = 1676561243 and user = 35968931 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • TomNicholas · 1 ✖

issue 1

  • Process getting killed due to high memory consumption of xarray's nbytes method · 1 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1516802286 https://github.com/pydata/xarray/issues/7772#issuecomment-1516802286 https://api.github.com/repos/pydata/xarray/issues/7772 IC_kwDOAMm_X85aaJDu TomNicholas 35968931 2023-04-20T18:58:48Z 2023-04-20T18:58:48Z MEMBER

Thanks for raising this @dabhicusp !

So why have that if block at line 396?

Because xarray can wrap many different type of numpy-like arrays, and for some of those types then the self.size * self.dtype.itemsize approach may not return the correct size. Think of a sparse matrix for example - its size in memory is designed to be much smaller than the size of the matrix would suggest. That's why in general we defer to the underlying array itself to tell us its size if it can (i.e. if it has a .nbytes attribute).

But you're not using an unusual type of array, you're just opening a netCDF file as a numpy array, in theory lazily. The memory usage you're seeing is not desired, so something weird must be happening in the .nbytes call. Going deeper into the stack at that point would be helpful.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Process getting killed due to high memory consumption of xarray's nbytes method 1676561243

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);