issue_comments

Table actions

4 rows where issue = 935062144 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

author_association 2

issue 1

  • UserWarning when wrapping pint & dask arrays together · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
872502631 https://github.com/pydata/xarray/issues/5559#issuecomment-872502631 https://api.github.com/repos/pydata/xarray/issues/5559 MDEyOklzc3VlQ29tbWVudDg3MjUwMjYzMQ== dcherian 2448579 2021-07-01T19:41:14Z 2021-07-01T19:41:14Z MEMBER

And the solution would be to just call the method on the duck array without any kind of type checking first?

Or np.method(array)

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   UserWarning when wrapping pint & dask arrays together 935062144
872499933 https://github.com/pydata/xarray/issues/5559#issuecomment-872499933 https://api.github.com/repos/pydata/xarray/issues/5559 MDEyOklzc3VlQ29tbWVudDg3MjQ5OTkzMw== TomNicholas 35968931 2021-07-01T19:35:43Z 2021-07-01T19:35:43Z MEMBER

So this is actually an xarray problem not a dask/pint problem? And the solution would be to just call the method on the duck array without any kind of type checking first?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   UserWarning when wrapping pint & dask arrays together 935062144
872477132 https://github.com/pydata/xarray/issues/5559#issuecomment-872477132 https://api.github.com/repos/pydata/xarray/issues/5559 MDEyOklzc3VlQ29tbWVudDg3MjQ3NzEzMg== dcherian 2448579 2021-07-01T18:56:41Z 2021-07-01T18:59:27Z MEMBER

Is it correct that xarray ends up calling dask.array.mean() on the pint.Quantity(dask.Array) object inside the DataArray?

Yes that's correct. See https://github.com/pydata/xarray/blob/c472f8a4c79f872edb9dcd7825f786ecb9aff5c0/xarray/core/duck_array_ops.py#L49-L51

It may be time to update this method since we now depend on a minimum numpy version that supports NEP-18.

cc @shoyer

EDIT: You get there from _create_nan_agg_method: https://github.com/pydata/xarray/blob/c472f8a4c79f872edb9dcd7825f786ecb9aff5c0/xarray/core/duck_array_ops.py#L347

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   UserWarning when wrapping pint & dask arrays together 935062144
872441814 https://github.com/pydata/xarray/issues/5559#issuecomment-872441814 https://api.github.com/repos/pydata/xarray/issues/5559 MDEyOklzc3VlQ29tbWVudDg3MjQ0MTgxNA== jthielen 3460034 2021-07-01T17:57:32Z 2021-07-01T18:05:19Z CONTRIBUTOR

Is it correct that xarray ends up calling dask.array.mean() on the pint.Quantity(dask.Array) object inside the DataArray? I took a guess at that since I can replicate what is happening inside the DataArray with

```python import dask.array as da

da = xr.DataArray([1,2,3], attrs={'units': 'metres'})

chunked = da.chunk(1).pint.quantify()

da.mean(chunked.variable._data) ```

Also, the Dask warning Passing an object to dask.array.from_array which is already a Dask collection. This can lead to unexpected behavior. is a big red flag that the Pint Quantity is making its way into Dask internals where it should not end up.

If so, I think this gets into a thorny issue with duck array handling in Dask. It was decided in https://github.com/dask/dask/pull/6393 that deliberately calling Dask array operations like elemwise (so, presumably by extension, blockwise and the reductions in dask.array.reductions like mean()) on a non-Dask array implies that the user wants to turn that array into a dask array. This get problematic, however, for upcast types like Pint Quantities that wrap Dask Arrays, since then you can get dask.Array(pint.Quantity(dask.Array)), which is what I think is going on here?

If this all checks out, I believe this becomes a Dask issue to improve upcast type/duck Dask array handling.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   UserWarning when wrapping pint & dask arrays together 935062144

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);