Code example

```python da = xr.DataArray(np.ones((10, 10))).chunk(2) indc = xr.DataArray(np.random.randint(0, 9, 10)).chunk(2)

This fails:

da[{'dim_1' : indc}].values ```

Problem description

Indexing with chunked arrays fails, whereas it's fine with "normal" arrays. In case the indices are the result of a lazy calculation, I would like to continue lazily.

Expected Output

I would expect an output just like in the "un-chunked" case:

``` da[{'dim_1' : indc.compute()}].values

Returns: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

```

Output of xr.show_versions()

Applying a function to a data array with dtype = complex returns one with dtype = float. It seems to work before commit 17b70caa6eafa062fd31e7f39334b3de922ff422.

MCVE Code Sample

```python import numpy as np import xarray as xr

def func(x): return np.sum(x ** 2)

da = xr.DataArray(np.arange(234).reshape(2,3,4)) da = da + 1j * da da = da.chunk(dict(dim_1=1))

da2 = xr.apply_ufunc( func, da, vectorize=True, dask="parallelized", output_dtypes=[da.dtype], )

assert da2.dtype == da.dtype, "wrong dtype" ```

Expected Output

da and da2 should both have the same dtype=complex.

Problem Description

To me it seems to me that the kwarg meta=None somehow causes dask to allocate a float array and ignore the dtype kwargs (which seems to be carried through correctly down to dask.array.blockwise.blockwise()) . I'm not familiar with the apply_ufing and the dask code, so I can't tell on which end the bug sits.

Versions

As @shoyer pointed out in https://github.com/pydata/xarray/issues/3237, nanargmax/min from numpy or dask should be used when not working on object arrays. Also, nanargmin/max were added to the nputils module so they should be using bottleneck if available.

• [x] Closes #3237
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Problem Description

While digging for #2511 I found that da.argmax() causes compute on a dask array in nanargmax(a, axis=None): https://github.com/pydata/xarray/blob/131f6022763d35edf461b11857d7c4ec6630b19d/xarray/core/nanops.py#L120 I feel like this shouldn't be the case as da.max() and da.data.argmax() don't compute and it renders the laziness useless.

MCVE Code Sample

```python In [1]: import numpy as np
...: import dask
...: import xarray as xr

In [2]: class Scheduler:
...: """ From: https://stackoverflow.com/questions/53289286/ """
...:
...: def init(self, max_computes=0):
...: self.max_computes = max_computes
...: self.total_computes = 0
...:
...: def call(self, dsk, keys, kwargs):
...: self.total_computes += 1
...: if self.total_computes > self.max_computes:
...: raise RuntimeError(
...: "Too many dask computations were scheduled: {}".format(
...: self.total_computes
...: )
...: )
...: return dask.get(dsk, keys, kwargs)
...:

In [3]: scheduler = Scheduler()

In [4]: with dask.config.set(scheduler=scheduler):
...: da = xr.DataArray(
...: np.random.rand(234).reshape((2, 3, 4)),
...: ).chunk(dict(dim_0=1))
...:
...: dim = da.dims[-1]
...:
...: dask_idcs = da.data.argmax(axis=-1) # Computes 0 times
...: print("Total number of computes: %d" % scheduler.total_computes)
...:
...: da.max(dim) # Computes 0 times
...: print("Total number of computes: %d" % scheduler.total_computes)
...:
...: da.argmax(dim) # Does compute
...: print("Total number of computes: %d" % scheduler.total_computes)
...:
Total number of computes: 0
Total number of computes: 0

RuntimeError Traceback (most recent call last)
<ipython-input-4-f95c8753dbe6> in <module>
12 print("Total number of computes: %d" % scheduler.total_computes)
13
---> 14 da.argmax(dim) # Does compute
15 print("Total number of computes: %d" % scheduler.total_computes)
16

~/src/xarray/xarray/core/common.py in wrapped_func(self, dim, axis, skipna, kwargs)
42 def wrapped_func(self, dim=None, axis=None, skipna=None, kwargs):
43 return self.reduce(
---> 44 func, dim, axis, skipna=skipna, allow_lazy=True, **kwargs
45 )
46

~/src/xarray/xarray/core/dataarray.py in reduce(self, func, dim, axis, keep_attrs, keepdims, kwargs)
2120 """
2121
-> 2122 var = self.variable.reduce(func, dim, axis, keep_attrs, keepdims, kwargs)
2123 return self._replace_maybe_drop_dims(var)
2124

~/src/xarray/xarray/core/variable.py in reduce(self, func, dim, axis, keep_attrs, keepdims, allow_lazy, kwargs)
1456 input_data = self.data if allow_lazy else self.values
1457 if axis is not None:
-> 1458 data = func(input_data, axis=axis, kwargs)
1459 else:
1460 data = func(input_data, **kwargs)

~/src/xarray/xarray/core/duck_array_ops.py in f(values, axis, skipna, kwargs)
279
280 try:
--> 281 return func(values, axis=axis, kwargs)
282 except AttributeError:
283 if isinstance(values, dask_array_type):

~/src/xarray/xarray/core/nanops.py in nanargmax(a, axis)
118 if mask is not None:
119 mask = mask.all(axis=axis)
--> 120 if mask.any():
121 raise ValueError("All-NaN slice encountered")
122 return res

/usr/lib/python3.7/site-packages/dask/array/core.py in bool(self)
1370 )
1371 else:
-> 1372 return bool(self.compute())
1373
1374 nonzero = bool # python 2

/usr/lib/python3.7/site-packages/dask/base.py in compute(self, kwargs)
173 dask.base.compute
174 """
--> 175 (result,) = compute(self, traverse=False, kwargs) 176 return result
177

/usr/lib/python3.7/site-packages/dask/base.py in compute(args, kwargs)
444 keys = [x.dask_keys() for x in collections]
445 postcomputes = [x.dask_postcompute() for x in collections]
--> 446 results = schedule(dsk, keys, kwargs)
447 return repack([f(r, a) for r, (f, a) in zip(results, postcomputes)])
448
```

Expected Output

None of the methods should actually compute: ``` python Total number of computes: 0
Total number of computes: 0
Total number of computes: 0
````

Output of xr.show_versions()

Hi all,

I prepared a little PR which could close #2243 and would allow for a IMO clean way of writing data with complex dtypes (and others).

What do you think?

TODOs:

• [X] Closes #2243
• [X] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Hi, since the last update of dask (to version 0.15.3), iterating over a groupby object and indexing using np.int64 fails, when the DataArray was opend with chunks from netcdf.

I'm using xarray version 0.9.6 and the 'h5netcdf'-engin for reading/writing.

To reproduce: ``` import xarray as xr arr = xr.DataArray(np.random.rand(2, 3, 4), dims=['one', 'two', 'three']) arr.to_netcdf('test.nc', engine='h5netcdf')

arr_disk = xr.open_dataarray('test.nc', engine='h5netcdf', chunks=dict(one=1))

This produces the error:

[g for g in arr_disk.groupby('one')]

/usr/lib/python3.6/site-packages/xarray/core/groupby.py in _iter_grouped(self) 296 """Iterate over each element in this group""" 297 for indices in self._group_indices: --> 298 yield self._obj.isel(**{self._group_dim: indices}) 299 300 def _infer_concat_args(self, applied_example):

/usr/lib/python3.6/site-packages/xarray/core/dataarray.py in isel(self, drop, indexers) 677 DataArray.sel 678 """ --> 679 ds = self._to_temp_dataset().isel(drop=drop, indexers) 680 return self._from_temp_dataset(ds) 681

/usr/lib/python3.6/site-packages/xarray/core/dataset.py in isel(self, drop, indexers) 1141 for name, var in iteritems(self._variables): 1142 var_indexers = dict((k, v) for k, v in indexers if k in var.dims) -> 1143 new_var = var.isel(var_indexers) 1144 if not (drop and name in var_indexers): 1145 variables[name] = new_var

/usr/lib/python3.6/site-packages/xarray/core/variable.py in isel(self, **indexers) 568 if dim in indexers: 569 key[i] = indexers[dim] --> 570 return self[tuple(key)] 571 572 def squeeze(self, dim=None):

/usr/lib/python3.6/site-packages/xarray/core/variable.py in getitem(self, key) 398 dims = tuple(dim for k, dim in zip(key, self.dims) 399 if not isinstance(k, integer_types)) --> 400 values = self._indexable_data[key] 401 # orthogonal indexing should ensure the dimensionality is consistent 402 if hasattr(values, 'ndim'):

/usr/lib/python3.6/site-packages/xarray/core/indexing.py in getitem(self, key) 496 value = value[(slice(None),) * axis + (subkey,)] 497 else: --> 498 value = self.array[key] 499 return value 500

/home/herter/.local/lib/python3.6/site-packages/dask/array/core.py in getitem(self, index) 1220 1221 from .slicing import normalize_index, slice_with_dask_array -> 1222 index2 = normalize_index(index, self.shape) 1223 1224 if any(isinstance(i, Array) for i in index2):

/home/herter/.local/lib/python3.6/site-packages/dask/array/slicing.py in normalize_index(idx, shape) 760 idx = idx + (slice(None),) * (len(shape) - n_sliced_dims) 761 if len([i for i in idx if i is not None]) > len(shape): --> 762 raise IndexError("Too many indices for array") 763 764 none_shape = []

IndexError: Too many indices for array

/home/herter/.local/lib/python3.6/site-packages/dask/array/slicing.py(762)normalize_index() 760 idx = idx + (slice(None),) * (len(shape) - n_sliced_dims) 761 if len([i for i in idx if i is not None]) > len(shape): --> 762 raise IndexError("Too many indices for array") 763 764 none_shape = [] ```

I'm getting the same error when doing arr_disk[np.int64(0)]

Thanks Uli