issue_comments

Table actions

5 rows where issue = 140291221 and user = 306380 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • mrocklin · 5 ✖

issue 1

  • dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf · 5 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
196935638 https://github.com/pydata/xarray/issues/793#issuecomment-196935638 https://api.github.com/repos/pydata/xarray/issues/793 MDEyOklzc3VlQ29tbWVudDE5NjkzNTYzOA== mrocklin 306380 2016-03-15T17:26:41Z 2016-03-15T17:26:41Z MEMBER 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf 140291221
195811381 https://github.com/pydata/xarray/issues/793#issuecomment-195811381 https://api.github.com/repos/pydata/xarray/issues/793 MDEyOklzc3VlQ29tbWVudDE5NTgxMTM4MQ== mrocklin 306380 2016-03-12T21:32:56Z 2016-03-12T21:32:56Z MEMBER

To be clear, we ran into the NetCDF: HDF error error when having multiple threads in the same process open-read-close many different files. I don't think there was any concurrent access of the same file. The problem went away when we switched to using processes rather than threads.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf 140291221
195573297 https://github.com/pydata/xarray/issues/793#issuecomment-195573297 https://api.github.com/repos/pydata/xarray/issues/793 MDEyOklzc3VlQ29tbWVudDE5NTU3MzI5Nw== mrocklin 306380 2016-03-11T22:13:28Z 2016-03-11T22:13:28Z MEMBER

Yes, my apologies for the typo.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf 140291221
195562924 https://github.com/pydata/xarray/issues/793#issuecomment-195562924 https://api.github.com/repos/pydata/xarray/issues/793 MDEyOklzc3VlQ29tbWVudDE5NTU2MjkyNA== mrocklin 306380 2016-03-11T21:29:46Z 2016-03-11T21:29:46Z MEMBER

Sure. I'm not proposing any particular approach. I'm just supporting your previous idea that maybe the problem is having too many open file handles. It would be good to check this before diving into threading or concurrency issues.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf 140291221
195557013 https://github.com/pydata/xarray/issues/793#issuecomment-195557013 https://api.github.com/repos/pydata/xarray/issues/793 MDEyOklzc3VlQ29tbWVudDE5NTU1NzAxMw== mrocklin 306380 2016-03-11T21:16:41Z 2016-03-11T21:16:41Z MEMBER

1024 might be a common open file handle limit. Some things to try to isolate the issue: 1. Try this with dask.set_globals(get=dask.async.get_sync) to turn off threading 2. Try just opening all of the files and see if the NetCDF error presents itself under normal operation

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.async.RuntimeError: NetCDF: HDF error on xarray to_netcdf 140291221

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);