issue_comments

Table actions

3 rows where issue = 481866516 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

author_association 2

issue 1

  • xr.DataArray.where sets valid points to nan when using several dask chunks · 3 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
883805046 https://github.com/pydata/xarray/issues/3225#issuecomment-883805046 https://api.github.com/repos/pydata/xarray/issues/3225 IC_kwDOAMm_X840rct2 stale[bot] 26384082 2021-07-21T01:00:55Z 2021-07-21T01:00:55Z NONE

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   xr.DataArray.where sets valid points to nan when using several dask chunks 481866516
522546004 https://github.com/pydata/xarray/issues/3225#issuecomment-522546004 https://api.github.com/repos/pydata/xarray/issues/3225 MDEyOklzc3VlQ29tbWVudDUyMjU0NjAwNA== climachine 20225454 2019-08-19T12:19:51Z 2019-08-19T12:19:51Z NONE

Thanks for looking into this! I agree this is an unexpected or at least under-documented behaviour of dask arrays.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   xr.DataArray.where sets valid points to nan when using several dask chunks 481866516
522353487 https://github.com/pydata/xarray/issues/3225#issuecomment-522353487 https://api.github.com/repos/pydata/xarray/issues/3225 MDEyOklzc3VlQ29tbWVudDUyMjM1MzQ4Nw== shoyer 1217238 2019-08-18T20:38:40Z 2019-08-18T20:38:40Z MEMBER

There isn't really a notion of "deep copying" a dask array. Dask assumes that everything you apply to a dask array is a pure function (though this isn't directly enforced), so if you map a mutating function over the blocks of a dask array you could potentially get undefined behavior (especially likely in the context of distributed computing). So when you tell xarray to deep copy a dask array, it currently just makes a normal copy.

I agree this a little counterintuitive, but it isn't obvious to me exactly what the right fix would look like. Perhaps we could start raising an error or warning in this case?

For your specific problem, the fix is to do the copy inside mapped function, e.g., python def set_fraction_randomly_to_nan(data, frac_missing): np.random.seed(0) data = data.copy() data[np.random.rand(*data.shape) < frac_missing] = np.nan return data

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   xr.DataArray.where sets valid points to nan when using several dask chunks 481866516

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);