issue_comments

Table actions

7 rows where author_association = "NONE" and user = 2783717 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, created_at (date), updated_at (date)

user 1

  • jcrist · 7 ✖

author_association 1

  • NONE · 7 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
511136084 https://github.com/pydata/xarray/issues/3041#issuecomment-511136084 https://api.github.com/repos/pydata/xarray/issues/3041 MDEyOklzc3VlQ29tbWVudDUxMTEzNjA4NA== jcrist 2783717 2019-07-13T16:38:14Z 2019-07-13T16:39:25Z NONE

Apparently in CPython this happens if you have a reference cycle.

AFAIK this is no longer true after Python 3.4.

I took a look with @delandmeterp and found that manually adding a gc.collect() (which collects all cycles) after deleting temp fixes the issue. The cycle was keeping the object around longer than expected, resulting in the lock error. It's not clear to me where the cycle is coming from (I'm not familiar with xarray internals), but this is indeed the issue.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   nc file locked by xarray after (double) da.compute() call 460254571
350303123 https://github.com/pydata/xarray/issues/1769#issuecomment-350303123 https://api.github.com/repos/pydata/xarray/issues/1769 MDEyOklzc3VlQ29tbWVudDM1MDMwMzEyMw== jcrist 2783717 2017-12-08T16:14:11Z 2017-12-08T16:14:11Z NONE

Not with the current design, no. However, most ma methods are idempotent, so calling ma.masked_invalid on an already masked array will result in no changes.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Extend to_masked_array to support dask MaskedArrays 280385592
259204793 https://github.com/pydata/xarray/issues/1093#issuecomment-259204793 https://api.github.com/repos/pydata/xarray/issues/1093 MDEyOklzc3VlQ29tbWVudDI1OTIwNDc5Mw== jcrist 2783717 2016-11-08T17:37:25Z 2016-11-08T17:37:25Z NONE

I'm not sure if I follow how this is a duck typing use case. I'd write this as a method, following your suggestion on SO:

Toward this end, it would be nice if xarray had something like dask.array's to_delayed method for converting a Dataset into an array of delayed datasets, which you could then lazily convert into DataFrame objects and do your computation.

Can you explain why you think this could benefit from collection duck typing?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Convert xarray dataset to dask dataframe or delayed objects 187872991
201529862 https://github.com/pydata/xarray/issues/804#issuecomment-201529862 https://api.github.com/repos/pydata/xarray/issues/804 MDEyOklzc3VlQ29tbWVudDIwMTUyOTg2Mg== jcrist 2783717 2016-03-25T21:49:03Z 2016-03-25T21:49:03Z NONE

I'm fine with the duck typing, and could make that work fairly easily. The mutation is a bit trickier though. I see two good options here for what b = dask.compute(a) would do with a being an xarray object: 1. b is a new object with the computation performed, a still contains a dask array (uncomputed). 2. b is the same object as a, and the internals of a are mutated to contain evaluated data.

I'm slightly partial to option 2, as it seems to mesh more with how xarray currently uses dask. Thoughts?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Compute multiple dask backed arrays at once 143551401
194026664 https://github.com/pydata/xarray/issues/783#issuecomment-194026664 https://api.github.com/repos/pydata/xarray/issues/783 MDEyOklzc3VlQ29tbWVudDE5NDAyNjY2NA== jcrist 2783717 2016-03-08T23:41:38Z 2016-03-08T23:41:38Z NONE

We're going to try and do a bugfix release shortly. Thanks for reporting the issue!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Array size changes following loading of numpy array 138332032
193995046 https://github.com/pydata/xarray/issues/783#issuecomment-193995046 https://api.github.com/repos/pydata/xarray/issues/783 MDEyOklzc3VlQ29tbWVudDE5Mzk5NTA0Ng== jcrist 2783717 2016-03-08T22:20:48Z 2016-03-08T22:21:28Z NONE

@pwolfram, thanks for the bug report. This unearthed a pretty bad bug in the slicing code of dask. Should be fixed in https://github.com/dask/dask/pull/1038. If you have the chance, can you pull this branch and see if it fixes your problem?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Array size changes following loading of numpy array 138332032
193618154 https://github.com/pydata/xarray/issues/783#issuecomment-193618154 https://api.github.com/repos/pydata/xarray/issues/783 MDEyOklzc3VlQ29tbWVudDE5MzYxODE1NA== jcrist 2783717 2016-03-08T05:43:16Z 2016-03-08T05:43:16Z NONE

I'll look at this tomorrow if you don't beat me to it :)

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Array size changes following loading of numpy array 138332032

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);