home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

6 rows where author_association = "MEMBER", issue = 591101988 and user = 14808389 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 1

  • keewis · 6 ✖

issue 1

  • FIX: correct dask array handling in _calc_idxminmax · 6 ✖

author_association 1

  • MEMBER · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
608440269 https://github.com/pydata/xarray/pull/3922#issuecomment-608440269 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODQ0MDI2OQ== keewis 14808389 2020-04-03T13:41:50Z 2020-04-03T13:41:50Z MEMBER

I'd say add a new entry. Also, I think we're all just reviewers so adding just your name should be fine.

{
    "total_count": 2,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 1,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988
608406175 https://github.com/pydata/xarray/pull/3922#issuecomment-608406175 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODQwNjE3NQ== keewis 14808389 2020-04-03T12:28:51Z 2020-04-03T12:28:51Z MEMBER

I think so? For now, xfail if use_dask and x.dtype.kind == "M"

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988
608391739 https://github.com/pydata/xarray/pull/3922#issuecomment-608391739 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODM5MTczOQ== keewis 14808389 2020-04-03T11:53:22Z 2020-04-03T12:02:30Z MEMBER

it seems you can't use argmin with dask arrays if the dtype is datetime64 (M8): ```python In [24]: time = np.asarray(pd.date_range("2019-07-17", periods=10)) ...: array = xr.DataArray( ...: time, ...: dims="x", ...: coords={"x": np.arange(time.size) * 4}, ...: ).chunk({}) ...: array
Out[24]: <xarray.DataArray (x: 10)> dask.array<xarray-\<this-array>, shape=(10,), dtype=datetime64[ns], chunksize=(10,), chunktype=numpy.ndarray> Coordinates: * x (x) int64 0 4 8 12 16 20 24 28 32 36

In [25]: array.compute().argmin(dim="x")
Out[25]: <xarray.DataArray ()> array(0)

In [26]: array.argmin(dim="x")

UFuncTypeError Traceback (most recent call last) <ipython-input-26-e665d5b1b9b4> in <module> ----> 1 array.argmin(dim="x")

.../xarray/core/common.py in wrapped_func(self, dim, axis, skipna, kwargs) 44 45 def wrapped_func(self, dim=None, axis=None, skipna=None, kwargs): ---> 46 return self.reduce(func, dim, axis, skipna=skipna, **kwargs) 47 48 else:

.../xarray/core/dataarray.py in reduce(self, func, dim, axis, keep_attrs, keepdims, kwargs) 2260 """ 2261 -> 2262 var = self.variable.reduce(func, dim, axis, keep_attrs, keepdims, kwargs) 2263 return self._replace_maybe_drop_dims(var) 2264

.../xarray/core/variable.py in reduce(self, func, dim, axis, keep_attrs, keepdims, allow_lazy, kwargs) 1573 1574 if axis is not None: -> 1575 data = func(input_data, axis=axis, kwargs) 1576 else: 1577 data = func(input_data, **kwargs)

.../xarray/core/duck_array_ops.py in f(values, axis, skipna, kwargs) 302 303 try: --> 304 return func(values, axis=axis, kwargs) 305 except AttributeError: 306 if not isinstance(values, dask_array_type):

.../xarray/core/duck_array_ops.py in f(args, kwargs) 45 else: 46 wrapped = getattr(eager_module, name) ---> 47 return wrapped(args, **kwargs) 48 49 else:

~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in wrapped(x, axis, split_every, out) 1002 1003 def wrapped(x, axis=None, split_every=None, out=None): -> 1004 return arg_reduction( 1005 x, chunk, combine, agg, axis, split_every=split_every, out=out 1006 )

~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in arg_reduction(x, chunk, combine, agg, axis, split_every, out) 980 tmp = Array(graph, name, chunks, dtype=x.dtype) 981 dtype = np.argmin([1]).dtype --> 982 result = _tree_reduce(tmp, agg, axis, False, dtype, split_every, combine) 983 return handle_out(out, result) 984

~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in _tree_reduce(x, aggregate, axis, keepdims, dtype, split_every, combine, name, concatenate, reduced_meta) 243 if concatenate: 244 func = compose(func, partial(_concatenate2, axes=axis)) --> 245 return partial_reduce( 246 func, 247 x,

~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in partial_reduce(func, x, split_every, keepdims, dtype, name, reduced_meta) 314 if is_arraylike(meta) and meta.ndim != len(out_chunks): 315 if len(out_chunks) == 0: --> 316 meta = meta.sum() 317 else: 318 meta = meta.reshape((0,) * len(out_chunks))

~/.conda/envs/xarray/lib/python3.8/site-packages/numpy/core/_methods.py in _sum(a, axis, dtype, out, keepdims, initial, where) 36 def _sum(a, axis=None, dtype=None, out=None, keepdims=False, 37 initial=_NoValue, where=True): ---> 38 return umr_sum(a, axis, dtype, out, keepdims, initial, where) 39 40 def _prod(a, axis=None, dtype=None, out=None, keepdims=False,

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]') `` I guess that's adask` bug?

Edit: you can reproduce it without xarray, so definitely a bug in dask:

MWE with only <tt>numpy</tt> / <tt>dask.array</tt> ```python In [32]: time = np.asarray(pd.date_range("2019-07-17", periods=10)) ...: np.argmin(da.from_array(time)) --------------------------------------------------------------------------- UFuncTypeError Traceback (most recent call last) <ipython-input-32-190cb901ff65> in <module> 1 time = np.asarray(pd.date_range("2019-07-17", periods=10)) ----> 2 np.argmin(da.from_array(time)) <__array_function__ internals> in argmin(*args, **kwargs) ~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/core.py in __array_function__(self, func, types, args, kwargs) 1348 if da_func is func: 1349 return handle_nonmatching_names(func, args, kwargs) -> 1350 return da_func(*args, **kwargs) 1351 1352 @property ~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in wrapped(x, axis, split_every, out) 1002 1003 def wrapped(x, axis=None, split_every=None, out=None): -> 1004 return arg_reduction( 1005 x, chunk, combine, agg, axis, split_every=split_every, out=out 1006 ) ~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in arg_reduction(x, chunk, combine, agg, axis, split_every, out) 980 tmp = Array(graph, name, chunks, dtype=x.dtype) 981 dtype = np.argmin([1]).dtype --> 982 result = _tree_reduce(tmp, agg, axis, False, dtype, split_every, combine) 983 return handle_out(out, result) 984 ~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in _tree_reduce(x, aggregate, axis, keepdims, dtype, split_every, combine, name, concatenate, reduced_meta) 243 if concatenate: 244 func = compose(func, partial(_concatenate2, axes=axis)) --> 245 return partial_reduce( 246 func, 247 x, ~/.conda/envs/xarray/lib/python3.8/site-packages/dask/array/reductions.py in partial_reduce(func, x, split_every, keepdims, dtype, name, reduced_meta) 314 if is_arraylike(meta) and meta.ndim != len(out_chunks): 315 if len(out_chunks) == 0: --> 316 meta = meta.sum() 317 else: 318 meta = meta.reshape((0,) * len(out_chunks)) ~/.conda/envs/xarray/lib/python3.8/site-packages/numpy/core/_methods.py in _sum(a, axis, dtype, out, keepdims, initial, where) 36 def _sum(a, axis=None, dtype=None, out=None, keepdims=False, 37 initial=_NoValue, where=True): ---> 38 return umr_sum(a, axis, dtype, out, keepdims, initial, where) 39 40 def _prod(a, axis=None, dtype=None, out=None, keepdims=False, UFuncTypeError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]') ``` </details>
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988
608341636 https://github.com/pydata/xarray/pull/3922#issuecomment-608341636 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODM0MTYzNg== keewis 14808389 2020-04-03T09:47:14Z 2020-04-03T09:55:35Z MEMBER

I'd put the dask tests directly below the idxmin / idxmax tests, decorated with requires_dask.

Edit: that's easy for test_dataarray.py, but not as much for test_dataset.py. Since Dataset delegates to the DataArray methods we might get away with not adding dask tests to Dataset, but I'm not really sure.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988
608331001 https://github.com/pydata/xarray/pull/3922#issuecomment-608331001 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODMzMTAwMQ== keewis 14808389 2020-04-03T09:24:39Z 2020-04-03T09:24:39Z MEMBER

:+1:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988
608322689 https://github.com/pydata/xarray/pull/3922#issuecomment-608322689 https://api.github.com/repos/pydata/xarray/issues/3922 MDEyOklzc3VlQ29tbWVudDYwODMyMjY4OQ== keewis 14808389 2020-04-03T09:06:42Z 2020-04-03T09:06:42Z MEMBER

The issue is that argmax produces a ndim - 1 array of indexes. In the case of 2D input data that would be 1D, so since indexing only allows to index with 1D arrays, your code works. For ndim > 2, we'd be trying to index with arrays with ndim > 1, so the indexing fails.

To get your 3D example (and potentially every N-D example) to work, simply fall back to the wrapped array's integer indexing (using coordarray.data): ```python In [2]: darray = da.from_array( ...: np.random.RandomState(0).randn(10 *20 * 30).reshape(10, 20, 30), ...: chunks=(1, 20, 30), # so we actually have multiple blocks ...: name='data_arr' ...: ) ...: array = xr.DataArray( ...: darray, ...: dims=["x", "y", 'z'], ...: coords={"x": np.arange(10), "y": np.arange(20), "z": np.arange(30)}, ...: ) ...: array Out[2]: <xarray.DataArray 'data_arr' (x: 10, y: 20, z: 30)> dask.array<data_arr, shape=(10, 20, 30), dtype=float64, chunksize=(1, 20, 30), chunktype=numpy.ndarray> Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 * z (z) int64 0 1 2 3 4 5 6 7 8 9 10 ... 20 21 22 23 24 25 26 27 28 29

In [3]: indx = array.argmin(dim='z', keep_attrs=True, skipna=False) ...: res = indx.copy( ...: data=indx.data.map_blocks( ...: lambda ind, coord: coord[(ind,)], ...: array.z.data, ...: dtype=array.z.dtype ...: ) ...: )

In [4]: res.compute() Out[4]: <xarray.DataArray 'data_arr' (x: 10, y: 20)> array([[20, 3, 3, 11, 20, 17, 3, 27, 24, 1, 7, 4, 22, 14, 7, 18, 5, 18, 7, 19], [10, 21, 25, 3, 15, 25, 28, 8, 10, 9, 13, 3, 24, 17, 19, 23, 12, 19, 19, 28], [ 1, 26, 10, 9, 16, 8, 17, 8, 6, 24, 28, 13, 23, 22, 26, 13, 28, 11, 6, 16], [ 6, 9, 26, 27, 1, 2, 21, 8, 10, 19, 14, 14, 20, 25, 24, 4, 18, 12, 20, 2], [22, 5, 12, 17, 13, 23, 23, 8, 27, 22, 1, 19, 26, 16, 12, 17, 19, 28, 8, 12], [20, 8, 25, 13, 4, 12, 23, 13, 27, 18, 15, 28, 10, 10, 0, 12, 5, 14, 5, 27], [29, 0, 19, 7, 15, 2, 8, 8, 13, 4, 12, 1, 7, 19, 14, 0, 3, 7, 12, 9], [ 9, 8, 4, 9, 17, 6, 7, 5, 29, 0, 15, 28, 22, 6, 24, 24, 20, 0, 24, 23], [ 1, 19, 12, 20, 4, 26, 5, 13, 21, 26, 25, 10, 5, 1, 11, 21, 6, 18, 4, 21], [15, 27, 13, 7, 25, 3, 14, 14, 17, 15, 11, 4, 16, 22, 22, 23, 0, 16, 26, 13]]) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 8 9 * y (y) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 `` Note that in this caseres == indxsincez = np.arange(30)`

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  FIX: correct dask array handling in _calc_idxminmax 591101988

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 105.221ms · About: xarray-datasette