issue_comments

Table actions

3 rows where author_association = "MEMBER", issue = 224553135 and user = 2448579 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

user 1

  • dcherian · 3 ✖

issue 1

  • slow performance with open_mfdataset · 3 ✖

author_association 1

  • MEMBER · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
756948150 https://github.com/pydata/xarray/issues/1385#issuecomment-756948150 https://api.github.com/repos/pydata/xarray/issues/1385 MDEyOklzc3VlQ29tbWVudDc1Njk0ODE1MA== dcherian 2448579 2021-01-08T19:20:51Z 2021-01-08T19:20:51Z MEMBER

setting parallel=True seg faults... I'm betting that is some quirk of my python environment, though.

This is important! Otherwise that timing scales with number of files. If you get that to work, then you can convert to a dask dataframe and keep things lazy.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   slow performance with open_mfdataset 224553135
756830228 https://github.com/pydata/xarray/issues/1385#issuecomment-756830228 https://api.github.com/repos/pydata/xarray/issues/1385 MDEyOklzc3VlQ29tbWVudDc1NjgzMDIyOA== dcherian 2448579 2021-01-08T15:52:49Z 2021-01-08T15:53:22Z MEMBER

@jameshalgren A lot of these issues have been fixed. Have you tried the advice here: https://xarray.pydata.org/en/stable/io.html#reading-multi-file-datasets?

If not, a reproducible example would help (I have access to Cheyenne). Let's also move this conversation to the "Discussions" forum: https://github.com/pydata/xarray/discussions

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   slow performance with open_mfdataset 224553135
685164626 https://github.com/pydata/xarray/issues/1385#issuecomment-685164626 https://api.github.com/repos/pydata/xarray/issues/1385 MDEyOklzc3VlQ29tbWVudDY4NTE2NDYyNg== dcherian 2448579 2020-09-01T22:21:58Z 2020-09-01T22:21:58Z MEMBER

This is the most up-to-date documentation on this issue: https://xarray.pydata.org/en/stable/io.html#reading-multi-file-datasets

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   slow performance with open_mfdataset 224553135

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);