issue_comments

Table actions

where issue = 420139027 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

These facets timed out: issue

user 1

  • shoyer · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
472145306 https://github.com/pydata/xarray/issues/2807#issuecomment-472145306 https://api.github.com/repos/pydata/xarray/issues/2807 MDEyOklzc3VlQ29tbWVudDQ3MjE0NTMwNg== shoyer 1217238 2019-03-12T19:21:26Z 2019-03-12T19:21:26Z MEMBER

I understand that there might be some challenges with returning xarray objects, but it seems like taking xarray objects should be very straightforward. Anything problematic about that?

This would probably be fine as an opt-in option. I'm a little worried that this would be a confusing model for users -- we don't have any other functions that work like this.

Maybe something similar would work here? Xarray would construct a dummy Xarray chunk, apply the user defined function onto that chunk, and then extrapolate metadata out from there somehow.

Yes, this is another possibility, though with xarray there is quite a bit of metadata to extrapolate! We do something similar already in groupby.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   can the callables of apply_ufunc + dask get a typed/labeled array 420139027
472129555 https://github.com/pydata/xarray/issues/2807#issuecomment-472129555 https://api.github.com/repos/pydata/xarray/issues/2807 MDEyOklzc3VlQ29tbWVudDQ3MjEyOTU1NQ== shoyer 1217238 2019-03-12T18:37:03Z 2019-03-12T18:37:03Z MEMBER

In the first version of apply_ufunc, I experimented with applying functions that take and return xarray objects.

The challenge is that with dask's lazy evaluation, we don't know the structure of the returned objects until after evaluating the wrapped functions. So we can't rebuild xarray objects unless we require redundantly specify all the coordinates and attributes from the return values.

The alternative would be to make a parallel but eagerly evaluated version of apply_ufunc, e.g., by calling compute on each chunk and then reassembling the result (e.g., ttps://github.com/pydata/xarray/pull/2616). The downside is that this will load your data into memory, but maybe that's acceptable for your purposes.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   can the callables of apply_ufunc + dask get a typed/labeled array 420139027

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);