home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

4 rows where issue = 374025325 and user = 13190237 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 1

  • ulijh · 4 ✖

issue 1

  • Array indexing with dask arrays · 4 ✖

author_association 1

  • CONTRIBUTOR 4
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
525634152 https://github.com/pydata/xarray/issues/2511#issuecomment-525634152 https://api.github.com/repos/pydata/xarray/issues/2511 MDEyOklzc3VlQ29tbWVudDUyNTYzNDE1Mg== ulijh 13190237 2019-08-28T08:12:13Z 2019-08-28T08:12:13Z CONTRIBUTOR

I think the problem is somewhere here:

https://github.com/pydata/xarray/blob/aaeea6250b89e3605ee1d1a160ad50d6ed657c7e/xarray/core/utils.py#L85-L103

I don't think pandas.Index can hold lazy arrays. Could there be a way around exploiting dask.dataframe indexing methods?

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Array indexing with dask arrays 374025325
522986699 https://github.com/pydata/xarray/issues/2511#issuecomment-522986699 https://api.github.com/repos/pydata/xarray/issues/2511 MDEyOklzc3VlQ29tbWVudDUyMjk4NjY5OQ== ulijh 13190237 2019-08-20T12:15:18Z 2019-08-20T18:52:49Z CONTRIBUTOR

Even though the example from above does work, sadly, the following does not: python import xarray as xr import dask.array as da import numpy as np da = xr.DataArray(np.random.rand(3*4*5).reshape((3,4,5))).chunk(dict(dim_0=1)) idcs = da.argmax('dim_2') da[dict(dim_2=idcs)] results in ``` python


TypeError Traceback (most recent call last) <ipython-input-4-3542cdd6d61c> in <module> ----> 1 da[dict(dim_2=idcs)]

~/src/xarray/xarray/core/dataarray.py in getitem(self, key) 604 else: 605 # xarray-style array indexing --> 606 return self.isel(indexers=self._item_key_to_dict(key)) 607 608 def setitem(self, key: Any, value: Any) -> None:

~/src/xarray/xarray/core/dataarray.py in isel(self, indexers, drop, **indexers_kwargs) 986 """ 987 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "isel") --> 988 ds = self._to_temp_dataset().isel(drop=drop, indexers=indexers) 989 return self._from_temp_dataset(ds) 990

~/src/xarray/xarray/core/dataset.py in isel(self, indexers, drop, **indexers_kwargs) 1901 indexes[name] = new_index 1902 else: -> 1903 new_var = var.isel(indexers=var_indexers) 1904 1905 variables[name] = new_var

~/src/xarray/xarray/core/variable.py in isel(self, indexers, drop, **indexers_kwargs) 984 if dim in indexers: 985 key[i] = indexers[dim] --> 986 return self[tuple(key)] 987 988 def squeeze(self, dim=None):

~/src/xarray/xarray/core/variable.py in getitem(self, key) 675 array x.values directly. 676 """ --> 677 dims, indexer, new_order = self._broadcast_indexes(key) 678 data = as_indexable(self._data)[indexer] 679 if new_order:

~/src/xarray/xarray/core/variable.py in _broadcast_indexes(self, key) 532 if isinstance(k, Variable): 533 if len(k.dims) > 1: --> 534 return self._broadcast_indexes_vectorized(key) 535 dims.append(k.dims[0]) 536 elif not isinstance(k, integer_types):

~/src/xarray/xarray/core/variable.py in _broadcast_indexes_vectorized(self, key) 660 new_order = None 661 --> 662 return out_dims, VectorizedIndexer(tuple(out_key)), new_order 663 664 def getitem(self, key):

~/src/xarray/xarray/core/indexing.py in init(self, key) 460 raise TypeError( 461 "unexpected indexer type for {}: {!r}".format( --> 462 type(self).name, k 463 ) 464 )

TypeError: unexpected indexer type for VectorizedIndexer: dask.array<arg_agg-aggregate, shape=(3, 4), dtype=int64, chunksize=(1, 4)> ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Array indexing with dask arrays 374025325
498178025 https://github.com/pydata/xarray/issues/2511#issuecomment-498178025 https://api.github.com/repos/pydata/xarray/issues/2511 MDEyOklzc3VlQ29tbWVudDQ5ODE3ODAyNQ== ulijh 13190237 2019-06-03T09:13:49Z 2019-06-03T09:13:49Z CONTRIBUTOR

As of version 0.12 indexing with dask arrays works out of the box... I think this can be closed now.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Array indexing with dask arrays 374025325
433304954 https://github.com/pydata/xarray/issues/2511#issuecomment-433304954 https://api.github.com/repos/pydata/xarray/issues/2511 MDEyOklzc3VlQ29tbWVudDQzMzMwNDk1NA== ulijh 13190237 2018-10-26T06:48:54Z 2018-10-26T06:48:54Z CONTRIBUTOR

It seem's working fine with the following change but it has a lot of dublicated code... ``` diff --git a/xarray/core/indexing.py b/xarray/core/indexing.py index d51da471..9fe93581 100644 --- a/xarray/core/indexing.py +++ b/xarray/core/indexing.py @@ -7,6 +7,7 @@ from datetime import timedelta

import numpy as np import pandas as pd +import dask.array as da

from . import duck_array_ops, nputils, utils from .pycompat import ( @@ -420,6 +421,19 @@ class VectorizedIndexer(ExplicitIndexer): 'have different numbers of dimensions: {}' .format(ndims)) k = np.asarray(k, dtype=np.int64) + elif isinstance(k, dask_array_type): + if not np.issubdtype(k.dtype, np.integer): + raise TypeError('invalid indexer array, does not have ' + 'integer dtype: {!r}'.format(k)) + if ndim is None: + ndim = k.ndim + elif ndim != k.ndim: + ndims = [k.ndim for k in key + if isinstance(k, (np.ndarray) + dask_array_type)] + raise ValueError('invalid indexer key: ndarray arguments ' + 'have different numbers of dimensions: {}' + .format(ndims)) + k = da.array(k, dtype=np.int64) else: raise TypeError('unexpected indexer type for {}: {!r}' .format(type(self).name, k)) ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Array indexing with dask arrays 374025325

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 570.352ms · About: xarray-datasette