home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 778329536

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/4898#issuecomment-778329536 https://api.github.com/repos/pydata/xarray/issues/4898 778329536 MDEyOklzc3VlQ29tbWVudDc3ODMyOTUzNg== 367900 2021-02-12T17:23:51Z 2021-02-12T17:23:51Z CONTRIBUTOR

A quick check with the debugger and it is the null_mask.any() call that is causing it to compute.

I think I've found another problem with _maybe_null_out if it is reducing over all dimensions. With the altered MCVE

```python import numpy as np import xarray as xr

def worker(da): if da.shape == (0, 0): return da

res = xr.full_like(da, np.nan)
res[0, 0] = 1
return res

da = xr.DataArray( np.random.normal(size=(20, 500)), dims=("x", "y"), coords=(np.arange(20), np.arange(500)), )

da = da.chunk(dict(x=5)) lazy = da.map_blocks(worker) result_allaxes = lazy.sum(skipna=True, min_count=5) result_allaxes.load() ```

I would expect result_allaxes to be nan since there are four blocks and therefore four non-nan values, less than min_count. Instead it is 4.

The problem seems to be the dtype check:

https://github.com/pydata/xarray/blob/5296ed18272a856d478fbbb3d3253205508d1c2d/xarray/core/nanops.py#L39

The test returns True for float64 and so the block isn't run. Another MCVE:

```python import numpy as np from xarray.core import dtypes

print(dtypes.NAT_TYPES) print(np.dtype("float64") in dtypes.NAT_TYPES) ```

Output: console (numpy.datetime64('NaT'), numpy.timedelta64('NaT')) True where I think False would be expected. Should I open a separate issue for this or can we track it here too?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  807089005
Powered by Datasette · Queries took 239.013ms · About: xarray-datasette