issue_comments

Table actions

6 rows where issue = 383057458 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • shoyer · 6 ✖

issue 1

  • Lock related problem in on travis-ci but not on local machine · 6 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
445944423 https://github.com/pydata/xarray/issues/2560#issuecomment-445944423 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0NTk0NDQyMw== shoyer 1217238 2018-12-10T19:37:31Z 2018-12-10T19:37:31Z MEMBER

See also https://github.com/pydata/xarray/issues/2535 for rasterio/netCDF4 issues.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458
445504270 https://github.com/pydata/xarray/issues/2560#issuecomment-445504270 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0NTUwNDI3MA== shoyer 1217238 2018-12-09T01:58:47Z 2018-12-09T01:58:47Z MEMBER

If either of you have time, it would be great if you could test out https://github.com/pydata/xarray/pull/2595 to see if that resolves your issue without requiring the environment variable.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458
445502216 https://github.com/pydata/xarray/issues/2560#issuecomment-445502216 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0NTUwMjIxNg== shoyer 1217238 2018-12-09T01:06:26Z 2018-12-09T01:06:26Z MEMBER

OK, I think I understand what's going on here and why this issue only appears with xarray v0.11.

The problem is related to how you are opening files with xarray but not closing them: https://github.com/limix/limix/blob/8bc0861035cc60b3ce7bcbf7f147bcc828580828/limix/qtl/test/test_qtl_xarr.py#L20

With v0.11, xarray introduced a least-recently-used cache for netCDF files. This means xarray's LRUCache is holding on to a reference to your files, so they never get garbage collected, and automatically closed by netCDF4-Python. Hence the HDF5 locks never get released.

Although it is indeed a good practice to always explicitly close files, I think we should explore using weak references to keep track of files in our cache so we don't hold on to them for holding than necessary.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458
445276713 https://github.com/pydata/xarray/issues/2560#issuecomment-445276713 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0NTI3NjcxMw== shoyer 1217238 2018-12-07T15:57:56Z 2018-12-07T15:57:56Z MEMBER

if you have a simpler example, that would make things easier to debug. The error you're seeing here is basically "HDF5 encountered an error" which could happen for any number of reasons.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458
440908787 https://github.com/pydata/xarray/issues/2560#issuecomment-440908787 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0MDkwODc4Nw== shoyer 1217238 2018-11-22T04:22:50Z 2018-11-22T04:22:50Z MEMBER

Could you share a link the file in which these tests are defined? I couldn't find it on the master branch.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458
440907957 https://github.com/pydata/xarray/issues/2560#issuecomment-440907957 https://api.github.com/repos/pydata/xarray/issues/2560 MDEyOklzc3VlQ29tbWVudDQ0MDkwNzk1Nw== shoyer 1217238 2018-11-22T04:16:16Z 2018-11-22T04:16:16Z MEMBER

Thanks for the report!

This might be an xarray issue, or it might also be a netCDF4 issue -- I wonder if you might have different versions of libnetcdf installed locally and on Traivs-CI?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Lock related problem in on travis-ci but not on local machine 383057458

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);