issue_comments

Table actions

4 rows where issue = 1260047355 and user = 1197350 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date)

user 1

  • rabernat · 4 ✖

issue 1

  • Obscure h5netcdf http serialization issue with python's http.server · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1146377099 https://github.com/pydata/xarray/issues/6662#issuecomment-1146377099 https://api.github.com/repos/pydata/xarray/issues/6662 IC_kwDOAMm_X85EVFOL rabernat 1197350 2022-06-03T21:30:48Z 2022-06-03T21:30:48Z MEMBER

Following up on the suggestion from @shoyer in to not use a context manager, if I redefine my function as

```python def open_pickle_and_reload(path): of = fsspec.open(path, mode='rb').open() ds1 = xr.open_dataset(of, engine='h5netcdf')

# pickle it and reload it
ds2 = loads(dumps(ds1))
ds2.load()

```

...it appears to work fine.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Obscure h5netcdf http serialization issue with python's http.server 1260047355
1146184372 https://github.com/pydata/xarray/issues/6662#issuecomment-1146184372 https://api.github.com/repos/pydata/xarray/issues/6662 IC_kwDOAMm_X85EUWK0 rabernat 1197350 2022-06-03T17:05:00Z 2022-06-03T17:06:26Z MEMBER

python with fsspec.open('http://127.0.0.1:8000/tiny.nc', mode='rb') as fp: with xr.open_dataset(fp, engine='h5netcdf') as ds1: print(type(fp)) print(fp.__dict__) ds1.load()

<class 'fsspec.implementations.http.HTTPFile'> {'asynchronous': False, 'url': 'http://127.0.0.1:8000/tiny.nc', 'session': <aiohttp.client.ClientSession object at 0x18bcdddc0>, '_details': {'name': 'http://127.0.0.1:8000/tiny.nc', 'size': 6164, 'type': 'file'}, 'size': 6164, 'path': 'http://127.0.0.1:8000/tiny.nc', 'fs': <fsspec.implementations.http.HTTPFileSystem object at 0x110059dc0>, 'mode': 'rb', 'blocksize': 5242880, 'loc': 1075, 'autocommit': True, 'end': None, 'start': None, '_closed': False, 'kwargs': {}, 'cache': <fsspec.caching.BytesCache object at 0x18eda16d0>, 'loop': <_UnixSelectorEventLoop running=True closed=False debug=False>}

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Obscure h5netcdf http serialization issue with python's http.server 1260047355
1146119478 https://github.com/pydata/xarray/issues/6662#issuecomment-1146119478 https://api.github.com/repos/pydata/xarray/issues/6662 IC_kwDOAMm_X85EUGU2 rabernat 1197350 2022-06-03T16:04:21Z 2022-06-03T16:05:40Z MEMBER

The http.server apparently does not accept range requests. That could definitely be related. However, I don't understand why that would affect only the pickled version. If the server doesn't support range requests, how are we able to load the file at all? This works:

python with fsspec.open('http://127.0.0.1:8000/tiny.nc', mode='rb') as fp: with xr.open_dataset(fp, engine='h5netcdf') as ds1: ds1.load()

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Obscure h5netcdf http serialization issue with python's http.server 1260047355
1146099479 https://github.com/pydata/xarray/issues/6662#issuecomment-1146099479 https://api.github.com/repos/pydata/xarray/issues/6662 IC_kwDOAMm_X85EUBcX rabernat 1197350 2022-06-03T15:54:34Z 2022-06-03T15:54:34Z MEMBER

Python's HTTP server does not normally provide content lengths without some extra work, that might be the difference.

Don't think that's it.

% curl -I "http://127.0.0.1:8000/tiny.nc" HTTP/1.0 200 OK Server: SimpleHTTP/0.6 Python/3.9.9 Date: Fri, 03 Jun 2022 15:53:52 GMT Content-type: application/x-netcdf Content-Length: 6164 Last-Modified: Fri, 03 Jun 2022 15:00:52 GMT

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Obscure h5netcdf http serialization issue with python's http.server 1260047355

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);