issue_comments

Table actions

5 rows where issue = 1152047670 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

author_association 2

issue 1

  • Read/Write performance optimizations for netcdf files · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1094133057 https://github.com/pydata/xarray/issues/6309#issuecomment-1094133057 https://api.github.com/repos/pydata/xarray/issues/6309 IC_kwDOAMm_X85BNyVB max-sixty 5635139 2022-04-09T22:16:15Z 2022-04-09T22:16:15Z MEMBER

Great, thanks @hmaarrfk — we'd definitely take a PR for the docs or something to pass along kwargs. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Read/Write performance optimizations for netcdf files 1152047670
1094086518 https://github.com/pydata/xarray/issues/6309#issuecomment-1094086518 https://api.github.com/repos/pydata/xarray/issues/6309 IC_kwDOAMm_X85BNm92 hmaarrfk 90008 2022-04-09T17:06:13Z 2022-04-09T17:06:13Z CONTRIBUTOR

@max-sixty unfortunately, I think the way hdf5 is designed, it doesn't try to be too smart about what would be the best fine tuning for your particular system. In some ways, this is the correct approach.

The current constructor pathway: https://github.com/pydata/xarray/blob/main/xarray/backends/h5netcdf_.py#L164

Doesn't provide a user with a catchall-kwargs. I think this would be an acceptable solution.

I should say that the the performance of the direct driver is terrible without aligned data: https://github.com/Unidata/netcdf-c/pull/2206#issuecomment-1054855769

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Read/Write performance optimizations for netcdf files 1152047670
1094081567 https://github.com/pydata/xarray/issues/6309#issuecomment-1094081567 https://api.github.com/repos/pydata/xarray/issues/6309 IC_kwDOAMm_X85BNlwf dcherian 2448579 2022-04-09T16:40:45Z 2022-04-09T16:40:45Z MEMBER

It would be nice to add this to our documentation. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Read/Write performance optimizations for netcdf files 1152047670
1094071783 https://github.com/pydata/xarray/issues/6309#issuecomment-1094071783 https://api.github.com/repos/pydata/xarray/issues/6309 IC_kwDOAMm_X85BNjXn max-sixty 5635139 2022-04-09T15:46:47Z 2022-04-09T15:46:47Z MEMBER

Thanks @hmaarrfk .

Something I didn't quite understand: what do the values in this map refer to:

kwargs = { "invalid_netcdf": invalid_netcdf, "phony_dims": phony_dims, "decode_vlen_strings": decode_vlen_strings, 'alignment_threshold': alignment_threshold, 'alignment_interval': alignment_interval, }

To what extent should this be done in xarray vs the underlying library? I wasn't sure how much context the underlying library needs to execute this well.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Read/Write performance optimizations for netcdf files 1152047670
1052386013 https://github.com/pydata/xarray/issues/6309#issuecomment-1052386013 https://api.github.com/repos/pydata/xarray/issues/6309 IC_kwDOAMm_X84-uiLd hmaarrfk 90008 2022-02-26T17:57:33Z 2022-02-26T17:57:33Z CONTRIBUTOR

I have to elaborate that this may be even more important for users that READ the data back alot. Reading with the standard Xarray operands hits other limits, but one limit that it definitely hits is that of the HDF5 driver used.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Read/Write performance optimizations for netcdf files 1152047670

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);