issue_comments

Table actions

2 rows where author_association = "MEMBER" and issue = 165104458 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • mfdataset fails at chunking after opening · 2 ✖

author_association 1

  • MEMBER · 2 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
232115059 https://github.com/pydata/xarray/issues/896#issuecomment-232115059 https://api.github.com/repos/pydata/xarray/issues/896 MDEyOklzc3VlQ29tbWVudDIzMjExNTA1OQ== shoyer 1217238 2016-07-12T17:17:38Z 2016-07-12T17:17:38Z MEMBER

@apatlpo Along what axis do your multiple files differ?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   mfdataset fails at chunking after opening 165104458
232101159 https://github.com/pydata/xarray/issues/896#issuecomment-232101159 https://api.github.com/repos/pydata/xarray/issues/896 MDEyOklzc3VlQ29tbWVudDIzMjEwMTE1OQ== shoyer 1217238 2016-07-12T16:28:40Z 2016-07-12T16:28:40Z MEMBER

This error indicates that Python is running out of memory. Dask (which we use with mfdataset) can help with that, but it doesn't always solve the issue.

How is the data arranged in each of the input files? What version of dask are you using? This might be an issue with dask.array not fusing calls to __getitem__ and loading entire files into memory.

cc @mrocklin @jcrist

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   mfdataset fails at chunking after opening 165104458

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);