issue_comments

Table actions

3 rows where issue = 439875798 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

author_association 2

issue 1

  • encoding of boolean dtype in zarr · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
652598966 https://github.com/pydata/xarray/issues/2937#issuecomment-652598966 https://api.github.com/repos/pydata/xarray/issues/2937 MDEyOklzc3VlQ29tbWVudDY1MjU5ODk2Ng== rabernat 1197350 2020-07-01T19:14:35Z 2020-07-01T19:14:35Z MEMBER

My approach here was to use compression and filters to minimize the on-disk storage. Here is my .zarray for the dataset in question

json { "chunks": [ 1, 13, 4320, 4320 ], "compressor": { "check": -1, "filters": [ { "dist": 1, "id": 3 }, { "id": 33, "preset": 1 } ], "format": 3, "id": "lzma", "preset": null }, "dtype": "|i1", "fill_value": null, "filters": null, "order": "C", "shape": [ 90, 13, 4320, 4320 ], "zarr_format": 2

This does not solve the in-memory problem, but that's a numpy issue.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   encoding of boolean dtype in zarr 439875798
652358007 https://github.com/pydata/xarray/issues/2937#issuecomment-652358007 https://api.github.com/repos/pydata/xarray/issues/2937 MDEyOklzc3VlQ29tbWVudDY1MjM1ODAwNw== joshmoore 88113 2020-07-01T11:16:40Z 2020-07-01T11:16:40Z NONE

@rabernat : I don't assume you've found a workaround for this?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   encoding of boolean dtype in zarr 439875798
489656124 https://github.com/pydata/xarray/issues/2937#issuecomment-489656124 https://api.github.com/repos/pydata/xarray/issues/2937 MDEyOklzc3VlQ29tbWVudDQ4OTY1NjEyNA== shoyer 1217238 2019-05-06T15:08:41Z 2019-05-06T15:08:41Z MEMBER

So maybe we make the boolean encoding optional?

Sounds good to me! We clearly don't need to be doing with data stored in zarr.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   encoding of boolean dtype in zarr 439875798

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);