issue_comments

Table actions

2 rows where issue = 289342234 and user = 102827 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • cchwala · 2 ✖

issue 1

  • HDF5 error when working with compressed NetCDF files and the dask multiprocessing scheduler · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
361532119 https://github.com/pydata/xarray/issues/1836#issuecomment-361532119 https://api.github.com/repos/pydata/xarray/issues/1836 MDEyOklzc3VlQ29tbWVudDM2MTUzMjExOQ== cchwala 102827 2018-01-30T09:32:26Z 2018-01-30T09:32:26Z CONTRIBUTOR

Thanks @jhamman for looking into this.

Currently I am fine with using persist() since I can break down my analysis workflow to certain time periods for which data fits into RAM on a large machine. As I have written, the distributed scheduler failed for me because of #1464. But I would like to use it in the future. From other discussions on the dask schedulers (here or on SO) using the distributed scheduler seems to be a general recommendation anyway.

In summary, I am fine with my current workaround. I do not think that solving this issue has a high priority, in particular when the distributed scheduler is further improved. The main annoyance was to track down the problem described in my first post. Hence, maybe the limitations of the schedulers could be described a bit better in the documentation. Would you want a PR on this?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   HDF5 error when working with compressed NetCDF files and the dask multiprocessing scheduler 289342234
358445479 https://github.com/pydata/xarray/issues/1836#issuecomment-358445479 https://api.github.com/repos/pydata/xarray/issues/1836 MDEyOklzc3VlQ29tbWVudDM1ODQ0NTQ3OQ== cchwala 102827 2018-01-17T21:07:43Z 2018-01-17T21:07:43Z CONTRIBUTOR

Thanks for the quick answer.

The problem is that my actual use case also involves writing back a xarray.Dataset via to_netcdf(). I left this out of the example above to isolate the problem. With the distributed scheduler and to_netcdf(), I ran into this issue #1464. As I can see, this might be fixed "soon" (#1793).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   HDF5 error when working with compressed NetCDF files and the dask multiprocessing scheduler 289342234

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);