issue_comments

Table actions

4 rows where issue = 597657663 and user = 6948919 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • bekatd · 4 ✖

issue 1

  • Hangs while saving netcdf file opened using xr.open_mfdataset with lock=None · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
778972571 https://github.com/pydata/xarray/issues/3961#issuecomment-778972571 https://api.github.com/repos/pydata/xarray/issues/3961 MDEyOklzc3VlQ29tbWVudDc3ODk3MjU3MQ== bekatd 6948919 2021-02-15T06:11:34Z 2021-02-15T06:11:34Z NONE

Please make some dummy tests, I did time.sleep, prior every operation. This was the only workaround that really worked. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Hangs while saving netcdf file opened using xr.open_mfdataset with lock=None 597657663
778839471 https://github.com/pydata/xarray/issues/3961#issuecomment-778839471 https://api.github.com/repos/pydata/xarray/issues/3961 MDEyOklzc3VlQ29tbWVudDc3ODgzOTQ3MQ== bekatd 6948919 2021-02-14T20:47:27Z 2021-02-14T20:47:27Z NONE

Is the current recommended solution to set lock=False and retry until success? Or, is it to keep lock=None and use zarr instead? @dcherian

Or alternatively you can try to set sleep between openings.

When you try to open same file from different functions with different operations, it is better to keep file opening function wrapped with a 1 second delay/sleep rather than direct open

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Hangs while saving netcdf file opened using xr.open_mfdataset with lock=None 597657663
730033698 https://github.com/pydata/xarray/issues/3961#issuecomment-730033698 https://api.github.com/repos/pydata/xarray/issues/3961 MDEyOklzc3VlQ29tbWVudDczMDAzMzY5OA== bekatd 6948919 2020-11-19T00:02:09Z 2020-11-19T00:04:13Z NONE

I have the same behaviour with MacOS (10.15). xarray=0.16.1, dask=2.30.0, netcdf4=1.5.4. Sometimes saves, sometimes doesn't. lock=False seems to work.

Lock false sometimes throws hd5 error. No clear solution.

The only solution I have found, sleep method for 1 second

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Hangs while saving netcdf file opened using xr.open_mfdataset with lock=None 597657663
690332310 https://github.com/pydata/xarray/issues/3961#issuecomment-690332310 https://api.github.com/repos/pydata/xarray/issues/3961 MDEyOklzc3VlQ29tbWVudDY5MDMzMjMxMA== bekatd 6948919 2020-09-10T14:36:53Z 2020-09-10T14:38:51Z NONE

Using:

  • xarray=0.16.0
  • dask=2.25.0
  • netcdf4=1.5.4

I am experiencing same when trying to write netcdf file using xr.to_netcdf() on a files opened via xr.open_mfdataset with lock=None.

Then I tried OP's suggestion and it worked like a charm

BUT

Now I am facing different issue. Seems that hdf5 IS NOT thread safe, since I encounter NetCDF: HDF error while applying different function on a netcdf file, previously were processed by another function with lock=False. script just terminates not even reaching any calculation step in the code. seems like lock=False works opposite and file is in a corrupted mode?

This is the BIGGEST issue and needs resolve ASAP

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Hangs while saving netcdf file opened using xr.open_mfdataset with lock=None 597657663

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);