issue_comments

Table actions

4 rows where issue = 295270362 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

These facets timed out: issue

user 1

  • shoyer · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
363948383 https://github.com/pydata/xarray/issues/1895#issuecomment-363948383 https://api.github.com/repos/pydata/xarray/issues/1895 MDEyOklzc3VlQ29tbWVudDM2Mzk0ODM4Mw== shoyer 1217238 2018-02-07T23:33:01Z 2018-02-07T23:33:01Z MEMBER

Ah, this may actually require a non-trivial amount of IO. It currently takes a non-trivial amount of time to read a zarr file. See pangeo-data/pangeo#99 (comment) . We're doing this on each deserialization?

We're unpickling the zarr objects. I don't know if that requires IO (probably not).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Avoid Adapters in task graphs? 295270362
363935874 https://github.com/pydata/xarray/issues/1895#issuecomment-363935874 https://api.github.com/repos/pydata/xarray/issues/1895 MDEyOklzc3VlQ29tbWVudDM2MzkzNTg3NA== shoyer 1217238 2018-02-07T22:40:22Z 2018-02-07T22:40:22Z MEMBER

What makes it expensive?

Well, presumably opening a zarr file requires a small amount of IO to read out the metadata.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Avoid Adapters in task graphs? 295270362
363931288 https://github.com/pydata/xarray/issues/1895#issuecomment-363931288 https://api.github.com/repos/pydata/xarray/issues/1895 MDEyOklzc3VlQ29tbWVudDM2MzkzMTI4OA== shoyer 1217238 2018-02-07T22:22:40Z 2018-02-07T22:22:40Z MEMBER

Do these objects happen to store any cached results? I'm seeing odd performance issues around these objects and am curious about any ways in which they might be fancy.

I don't think there's any caching here. All of these objects are stateless, though ZarrArrayWrapper does point back to a ZarrStore object and a zarr.Group object.

Any concerns about recreating these objects for every access?

No, not particularly, though potentially opening a zarr store could be a little expensive. I'm mostly not sure how this would be done. Currently, we open files, create array objects, do some lazy decoding and then create dask arrays with from_array.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Avoid Adapters in task graphs? 295270362
363921064 https://github.com/pydata/xarray/issues/1895#issuecomment-363921064 https://api.github.com/repos/pydata/xarray/issues/1895 MDEyOklzc3VlQ29tbWVudDM2MzkyMTA2NA== shoyer 1217238 2018-02-07T21:44:33Z 2018-02-07T21:44:33Z MEMBER

In principle this is fine, especially if this object is cheap to serialize, move, and deserialize.

Yes, that should be the case here. Each of these array objects is very lightweight and should be quickly pickled/unpickled.

On the other hand, once evaluated these do correspond to a large chunk of data (entire arrays). If this future needs to be evaluated before being passed around that would be a problem. Getitem fusing is pretty essential here for performance.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Avoid Adapters in task graphs? 295270362

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);