issue_comments

Table actions

7 rows where author_association = "MEMBER" and issue = 634869703 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

issue 1

  • Why am I able to load data from a closed dataset? · 7 ✖

author_association 1

  • MEMBER · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
649976034 https://github.com/pydata/xarray/issues/4131#issuecomment-649976034 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0OTk3NjAzNA== shoyer 1217238 2020-06-26T05:24:50Z 2020-06-26T05:24:50Z MEMBER

A user might expect that they can write something like: python with nc.Dataset("tmp_example.nc") as ncfile: dstore = xr.backends.NetCDF4DataStore(ncfile) with xr.open_dataset(dstore) as data: ...

Currently this raises an error when the outer context manager exits.

So maybe this is a case for not closing files that were not opened by xarray.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
649975829 https://github.com/pydata/xarray/issues/4131#issuecomment-649975829 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0OTk3NTgyOQ== kmuehlbauer 5821660 2020-06-26T05:24:15Z 2020-06-26T05:24:15Z MEMBER

@rsokl Yes, you're right for your issue. In my comment above I was referring to the case where the file was opened outside xarray.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
649973124 https://github.com/pydata/xarray/issues/4131#issuecomment-649973124 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0OTk3MzEyNA== shoyer 1217238 2020-06-26T05:16:38Z 2020-06-26T05:16:38Z MEMBER

Given the ambiguity, I think it would be reasonable to raise an error when trying to close a file not opened by xarray.

This would be a breaking change, so we if we change this we would need to start by issuing a warning.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
649972764 https://github.com/pydata/xarray/issues/4131#issuecomment-649972764 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0OTk3Mjc2NA== kmuehlbauer 5821660 2020-06-26T05:15:28Z 2020-06-26T05:15:28Z MEMBER

It isn't obvious to me what the right behavior is here. I can see a case for either closing the file (the user did request it to be closed, after all) or leaving it open.

Sure, it's heavily depending on use case. But good to know what and where to look for in case anything unusual is going on.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
649971441 https://github.com/pydata/xarray/issues/4131#issuecomment-649971441 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0OTk3MTQ0MQ== shoyer 1217238 2020-06-26T05:12:03Z 2020-06-26T05:12:03Z MEMBER

Nevertheless it seems unusual, that opening a file outside xarray will be closed inside xarray. At least I would not expect it, to be closed.

It isn't obvious to me what the right behavior is here. I can see a case for either closing the file (the user did request it to be closed, after all) or leaving it open.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
641750522 https://github.com/pydata/xarray/issues/4131#issuecomment-641750522 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0MTc1MDUyMg== kmuehlbauer 5821660 2020-06-10T06:18:36Z 2020-06-10T06:18:36Z MEMBER

This doesn't work when it is opened via netCDF4 and NetCDF4DataStore.

python import netCDF4 as nc import xarray as xr ncfile = nc.Dataset("tmp_example.nc") dstore = xr.backends.NetCDF4DataStore(ncfile) data = xr.open_dataset(dstore) print(ncfile.isopen()) data.close() print(ncfile.isopen()) print(data.foo)

``` True False /home/miniconda/envs/xarray/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in _getitem(self, key) 83 with self.datastore.lock: 84 original_array = self.get_array(needs_lock=False) ---> 85 array = getitem(original_array, key) 86 except IndexError: 87 # Catch IndexError in netCDF4 and return a more informative

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.getitem()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.shape.get()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._getdims()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()

RuntimeError: NetCDF: Not a valid ID ```

Also in @rsokl's case the data._file_obj is None after closing. First I thought that caching might be involved, but it doesn't seem the case.

Nevertheless it seems unusual, that opening a file outside xarray will be closed inside xarray. At least I would not expect it, to be closed. Should I open another issue for this, @shoyer? 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703
641602748 https://github.com/pydata/xarray/issues/4131#issuecomment-641602748 https://api.github.com/repos/pydata/xarray/issues/4131 MDEyOklzc3VlQ29tbWVudDY0MTYwMjc0OA== shoyer 1217238 2020-06-09T21:55:36Z 2020-06-09T21:55:36Z MEMBER

Does querying a closed dataset automatically reopen it?

That's exactly right.

We could conceivably raise an error instead in these situations, since this is probably a case of user error.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Why am I able to load data from a closed dataset? 634869703

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);