home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

5 rows where author_association = "CONTRIBUTOR", issue = 659129613 and user = 1610850 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • jacobtomlinson · 5 ✖

issue 1

  • Add ability to change underlying array type · 5 ✖

author_association 1

  • CONTRIBUTOR · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
660192634 https://github.com/pydata/xarray/issues/4234#issuecomment-660192634 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE5MjYzNA== jacobtomlinson 1610850 2020-07-17T16:07:31Z 2020-07-17T16:08:34Z CONTRIBUTOR

Those as_ methods sounds good. Would as_dense be the same as as_numpy?

Yeah it is possible to read direct to GPU from storage with GDS. We've experimented a little with zarr, I expect if something like zarr got GDS support and a zarr dataset was configured to use GDS then xr.open_zarr('zarr/path') would be already backed by cupy because the zarr internal array would be cupy.

Re: index variables.Can we avoid this for now?

I think that would be best.

However I did run into issues when trying to run the Compare weighted and unweighted mean temperature example with cupy. In that example the weights data array is generated from the latitude index. So the resulting DataArray is backed by numpy. I would expect that if it were a cuDF Index it would end up as a cupy data array.

In my testing I just cast the weights data array to cupy and carried on. So perhaps with this change users will just need to sprinkle some weights = weights.as_cupy() or weights = weights.asarray(cp.ndarray) type calls throughout their code when they need to.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660154476 https://github.com/pydata/xarray/issues/4234#issuecomment-660154476 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE1NDQ3Ng== jacobtomlinson 1610850 2020-07-17T14:58:53Z 2020-07-17T14:58:53Z CONTRIBUTOR

The only things I can think of that would make sense initially in an accessor would be to_cupy/from_cupy to move back and forth from the GPU, plus maybe an alias for get to from_cupy to match the cupy API.

An accessor does seem like a reasonable place to put that logic, but it also seems like a tiny amount of code to make, ship and maintain a separate package for. Plus those methods will either need to be used or duplicated in the core codebase to support things like plotting.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660148433 https://github.com/pydata/xarray/issues/4234#issuecomment-660148433 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE0ODQzMw== jacobtomlinson 1610850 2020-07-17T14:48:01Z 2020-07-17T14:54:41Z CONTRIBUTOR

This does sound like an option. However there are many situations within xarray where we need explicit cupy logic. Converting back to numpy before plotting is one example. I don't think that kind of logic can live in an accessor.

Unless you expect users to do something like this.

```python import xarray as xr import cupy as cp

ds = xr.tutorial.load_dataset("air_temperature") gds = ds.cupy.to_cupy()

Do some manipulation on the GPU

Grab a time slice

time_slice = gds.air.isel(time=0)

time_slice.cupy.to_numpy().plot() # I would hope that time_slice.plot() would work ```

I would be tempted to say that cupy is more like dask in that it is trying to implement the numpy array interface exactly but in a different paradigm (distributed, GPU, etc). And of course there are limitations and divergences because of the different paradigm. However it's not the same as pint which is trying to extend numpy and add more functionality.

So it makes sense to me that accessors for pint exist to add this extra functionality to xarray. But at least in theory cupy should be a drop-in replacement for numpy. So I don't expect a huge amount of logic will live in an accessor, compared to the amount of compatibility code that will need to go into xarray itself.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660136749 https://github.com/pydata/xarray/issues/4234#issuecomment-660136749 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDEzNjc0OQ== jacobtomlinson 1610850 2020-07-17T14:26:35Z 2020-07-17T14:32:03Z CONTRIBUTOR

@jthielen Something like this?

```python import xarray as xr import cupy as cp

@xr.register_dataset_accessor("cupy") class CupyAccessor: def to_cupy(self): """Convert all data arrays to cupy."""

    for var in self.data_vars: 
        self.data_vars[var].data = cp.asarray(self.data_vars[var].data)

    return self

```

Which would then be used like this.

```python import xarray as xr import cupy as cp

ds = xr.open_mfdataset("/path/to/files/*.nc") gds = ds.cupy.to_cupy() ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660030480 https://github.com/pydata/xarray/issues/4234#issuecomment-660030480 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDAzMDQ4MA== jacobtomlinson 1610850 2020-07-17T10:37:46Z 2020-07-17T10:37:46Z CONTRIBUTOR

cc @quasiben

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 15.354ms · About: xarray-datasette