issue_comments

Table actions

3 rows where issue = 166287789 and user = 1217238 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date)

user 1

  • shoyer · 3 ✖

issue 1

  • Pickle and .value vs. dask backend · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
249435675 https://github.com/pydata/xarray/issues/902#issuecomment-249435675 https://api.github.com/repos/pydata/xarray/issues/902 MDEyOklzc3VlQ29tbWVudDI0OTQzNTY3NQ== shoyer 1217238 2016-09-25T17:54:43Z 2016-09-25T17:54:43Z MEMBER

@crusaderky Let's just disable caching for dask.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Pickle and .value vs. dask backend 166287789
239937280 https://github.com/pydata/xarray/issues/902#issuecomment-239937280 https://api.github.com/repos/pydata/xarray/issues/902 MDEyOklzc3VlQ29tbWVudDIzOTkzNzI4MA== shoyer 1217238 2016-08-15T21:38:16Z 2016-08-15T21:38:16Z MEMBER

This is where you can find the core caching logic on Variable objects: https://github.com/pydata/xarray/blob/56abba54ca4e89af570fd9cacc8f3ffcf5a5c4c7/xarray/core/variable.py#L257-L305

Here's where we define load on Dataset and DataArray: https://github.com/pydata/xarray/blob/56abba54ca4e89af570fd9cacc8f3ffcf5a5c4c7/xarray/core/dataset.py#L305-L327 https://github.com/pydata/xarray/blob/56abba54ca4e89af570fd9cacc8f3ffcf5a5c4c7/xarray/core/dataarray.py#L523-L536

As I mentioned before, let's add .compute() to evaluate and return a new object, and use it for .values instead of caching. .load() can remain unchanged for when users actually want to cache data. And we can definitely disable automatically loading data in pickles.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Pickle and .value vs. dask backend 166287789
233692572 https://github.com/pydata/xarray/issues/902#issuecomment-233692572 https://api.github.com/repos/pydata/xarray/issues/902 MDEyOklzc3VlQ29tbWVudDIzMzY5MjU3Mg== shoyer 1217238 2016-07-19T16:40:39Z 2016-07-19T16:40:39Z MEMBER

I agree about loading data into memory automatically -- this behavior made sense before we used dask in xarray, but now it doesn't really.

We actually already have a .load() method for explicitly loading data into memory, though it might make sense to add .compute() as an alias, possibly without modifying the original dataset inplace.

I'm a little less certain about how to handle pickling data, because anytime you open a file from disk using open_dataset it's not going to pickle. But on the other hand, it's also not hard to explicitly write .load() or .compute() before using pickle or invoking multiprocessing.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Pickle and .value vs. dask backend 166287789

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);