issue_comments

Table actions

6 rows where issue = 522519084 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

author_association 3

issue 1

  • Allow nested dictionaries in the Zarr backend (#3517) · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1410658476 https://github.com/pydata/xarray/pull/3526#issuecomment-1410658476 https://api.github.com/repos/pydata/xarray/issues/3526 IC_kwDOAMm_X85UFPCs jhamman 2443309 2023-01-31T16:08:36Z 2023-01-31T16:08:36Z MEMBER

Bring back this old issue since I think there is value in getting something like this in. My suggestion would be to write a zarr-specific _validate_attrs function and use that, rather than stubbing in a special case for zarr in the generic one. @eddienko , I know this has sat for a long time but would you like to try to finish this up? 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084
554497403 https://github.com/pydata/xarray/pull/3526#issuecomment-554497403 https://api.github.com/repos/pydata/xarray/issues/3526 MDEyOklzc3VlQ29tbWVudDU1NDQ5NzQwMw== eddienko 3780274 2019-11-15T19:32:33Z 2019-11-15T19:32:33Z NONE

mmm, I am not sure I understand. Currently

ds = xr.Dataset() ds.attrs['foo'] = {'bar': 1} ds.to_zarr('foo.zarr', mode='w')

does not work. This is because dict is not allowed in the list of types that the attributes are validated against. My feeling is this should not produce an error.

This PR just addresses that so the code above works. The following works as well:

ds = xr.open_zarr('foo.zarr') ds.attrs['foo'] {'bar': 1}

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084
554479941 https://github.com/pydata/xarray/pull/3526#issuecomment-554479941 https://api.github.com/repos/pydata/xarray/issues/3526 MDEyOklzc3VlQ29tbWVudDU1NDQ3OTk0MQ== dcherian 2448579 2019-11-15T18:41:43Z 2019-11-15T18:41:43Z MEMBER

Whatever value is given then gets serialised using the built-in json.dumps. This means I believe that if someone provides a dict as an attribute value then that will get serialised as a JSON object,

@eddienko I think this means we can avoid converting to a list?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084
554463832 https://github.com/pydata/xarray/pull/3526#issuecomment-554463832 https://api.github.com/repos/pydata/xarray/issues/3526 MDEyOklzc3VlQ29tbWVudDU1NDQ2MzgzMg== alimanfoo 703554 2019-11-15T17:57:42Z 2019-11-15T17:57:42Z CONTRIBUTOR

FWIW in the Zarr Python implementation I don't think we do any special encoding or decoding of attribute values. Whatever value is given then gets serialised using the built-in json.dumps. This means I believe that if someone provides a dict as an attribute value then that will get serialised as a JSON object, and get deserialised back to a dict, although this is not something we test for currently.

From the zarr v2 spec point of view I think anything goes in the .zattrs file, as long as .zattrs is a JSON object at the root.

Hth.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084
554045517 https://github.com/pydata/xarray/pull/3526#issuecomment-554045517 https://api.github.com/repos/pydata/xarray/issues/3526 MDEyOklzc3VlQ29tbWVudDU1NDA0NTUxNw== jakirkham 3019665 2019-11-14T19:37:13Z 2019-11-14T19:37:13Z NONE

Yeah this probably works as these are just JSON files. That said, IDK that we are making any attempt to ensure this works. IOW I don't think this is tested or in the spec.

Additionally IDK that we do the same decoding on nested dictionaries as would be done on a flat dictionary. Meaning non-JSON values like datetime64/timedelta64 might not be handled correctly in this case.

Could be wrong about these things. Those are just my immediate thoughts.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084
553995368 https://github.com/pydata/xarray/pull/3526#issuecomment-553995368 https://api.github.com/repos/pydata/xarray/issues/3526 MDEyOklzc3VlQ29tbWVudDU1Mzk5NTM2OA== rabernat 1197350 2019-11-14T17:34:06Z 2019-11-14T17:34:06Z MEMBER

I am slammed right now with a proposal and can't really spend time on this, so I'll tag @jakirkham and @alimanfoo who can provide a zarr perspective on what is going on here.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Allow nested dictionaries in the Zarr backend (#3517) 522519084

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);