home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 665023246

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
665023246 MDU6SXNzdWU2NjUwMjMyNDY= 4263 apply_ufunc does not pass block_info to the applied function 1649961 closed 0     2 2020-07-24T08:53:51Z 2021-10-18T16:11:28Z 2021-10-18T16:11:28Z NONE      

What happened:

xarray.apply_ufunc does not pass block_info to the applied function when dask="parallelized".

What you expected to happen:

The applied function gets the special block_info argument (see the map_blocks documentation).

Minimal Complete Verifiable Example:

```python import sys

import dask.array import xarray

def f(a, block_info=None): loc = None if block_info is None else block_info[None]["chunk-location"] sys.stdout.write(f"chunk-location={loc}\n") return a

a = xarray.DataArray(dask.array.arange(24).reshape((4, 6)).rechunk((2, 3)), dims=("a", "b")) meta = dask.array.empty((0,), dtype=a.dtype)

xarray.apply_ufunc(f, a, dask="parallelized", output_dtypes=[a.dtype], meta=meta).compute();

chunk-location=None

chunk-location=None

chunk-location=None

chunk-location=None

dask.array.map_blocks(f, a.data, dtype=a.dtype, meta=meta).compute();

chunk-location=(0, 0)

chunk-location=(0, 1)

chunk-location=(1, 0)

chunk-location=(1, 1)

```

Anything else we need to know?:

It seems that xarray.apply_ufunc calls dask.array.blockwise directly, while block_info is handled by dask.array.map_blocks.

Environment:

Output of <tt>xr.show_versions()</tt>
INSTALLED VERSIONS
------------------
commit: None
python: 3.7.7 (default, May  6 2020, 04:59:01) 
[Clang 4.0.1 (tags/RELEASE_401/final)]
python-bits: 64
OS: Darwin
OS-release: 19.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: None
LOCALE: en_GB.UTF-8
libhdf5: 1.10.6
libnetcdf: None
xarray: 0.16.0
pandas: 1.0.5
numpy: 1.18.5
scipy: 1.5.0
netCDF4: None
pydap: None
h5netcdf: None
h5py: 2.10.0
Nio: None
zarr: 2.3.2
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.20.0
distributed: 2.20.0
matplotlib: 3.2.2
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.2.0.post20200714
pip: 20.1.1
conda: None
pytest: 5.4.3
IPython: 7.16.1
sphinx: 3.1.1
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4263/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 2 rows from issue in issue_comments
Powered by Datasette · Queries took 0.625ms · About: xarray-datasette