home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

9 rows where author_association = "CONTRIBUTOR" and issue = 659129613 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 2

  • jacobtomlinson 5
  • jthielen 4

issue 1

  • Add ability to change underlying array type · 9 ✖

author_association 1

  • CONTRIBUTOR · 9 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
660192634 https://github.com/pydata/xarray/issues/4234#issuecomment-660192634 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE5MjYzNA== jacobtomlinson 1610850 2020-07-17T16:07:31Z 2020-07-17T16:08:34Z CONTRIBUTOR

Those as_ methods sounds good. Would as_dense be the same as as_numpy?

Yeah it is possible to read direct to GPU from storage with GDS. We've experimented a little with zarr, I expect if something like zarr got GDS support and a zarr dataset was configured to use GDS then xr.open_zarr('zarr/path') would be already backed by cupy because the zarr internal array would be cupy.

Re: index variables.Can we avoid this for now?

I think that would be best.

However I did run into issues when trying to run the Compare weighted and unweighted mean temperature example with cupy. In that example the weights data array is generated from the latitude index. So the resulting DataArray is backed by numpy. I would expect that if it were a cuDF Index it would end up as a cupy data array.

In my testing I just cast the weights data array to cupy and carried on. So perhaps with this change users will just need to sprinkle some weights = weights.as_cupy() or weights = weights.asarray(cp.ndarray) type calls throughout their code when they need to.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660156875 https://github.com/pydata/xarray/issues/4234#issuecomment-660156875 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE1Njg3NQ== jthielen 3460034 2020-07-17T15:03:01Z 2020-07-17T15:03:01Z CONTRIBUTOR

The accessors could also be where wrapped xarray methods could go (so in your previous example, it could be time_slice.cupy.plot() to handle the NumPy conversion internally), but based on your explanations I'd agree that most of this makes more sense to be compatibility code that goes in xarray itself.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660154476 https://github.com/pydata/xarray/issues/4234#issuecomment-660154476 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE1NDQ3Ng== jacobtomlinson 1610850 2020-07-17T14:58:53Z 2020-07-17T14:58:53Z CONTRIBUTOR

The only things I can think of that would make sense initially in an accessor would be to_cupy/from_cupy to move back and forth from the GPU, plus maybe an alias for get to from_cupy to match the cupy API.

An accessor does seem like a reasonable place to put that logic, but it also seems like a tiny amount of code to make, ship and maintain a separate package for. Plus those methods will either need to be used or duplicated in the core codebase to support things like plotting.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660148433 https://github.com/pydata/xarray/issues/4234#issuecomment-660148433 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE0ODQzMw== jacobtomlinson 1610850 2020-07-17T14:48:01Z 2020-07-17T14:54:41Z CONTRIBUTOR

This does sound like an option. However there are many situations within xarray where we need explicit cupy logic. Converting back to numpy before plotting is one example. I don't think that kind of logic can live in an accessor.

Unless you expect users to do something like this.

```python import xarray as xr import cupy as cp

ds = xr.tutorial.load_dataset("air_temperature") gds = ds.cupy.to_cupy()

Do some manipulation on the GPU

Grab a time slice

time_slice = gds.air.isel(time=0)

time_slice.cupy.to_numpy().plot() # I would hope that time_slice.plot() would work ```

I would be tempted to say that cupy is more like dask in that it is trying to implement the numpy array interface exactly but in a different paradigm (distributed, GPU, etc). And of course there are limitations and divergences because of the different paradigm. However it's not the same as pint which is trying to extend numpy and add more functionality.

So it makes sense to me that accessors for pint exist to add this extra functionality to xarray. But at least in theory cupy should be a drop-in replacement for numpy. So I don't expect a huge amount of logic will live in an accessor, compared to the amount of compatibility code that will need to go into xarray itself.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660148938 https://github.com/pydata/xarray/issues/4234#issuecomment-660148938 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE0ODkzOA== jthielen 3460034 2020-07-17T14:48:59Z 2020-07-17T14:48:59Z CONTRIBUTOR

@jacobtomlinson Makes complete sense! Just wanted to make sure the option was considered as a possibility.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660142143 https://github.com/pydata/xarray/issues/4234#issuecomment-660142143 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDE0MjE0Mw== jthielen 3460034 2020-07-17T14:36:40Z 2020-07-17T14:36:40Z CONTRIBUTOR

@jacobtomlinson Indeed! A similar to_cupy could also be made for existing DataArrays on a DataArray accessor, and any other CuPy-related attributes/methods could be attached to the same accessors as well.

Though, to have those accessors be registered with just importing xarray and cupy, they would of course need to live in either xarray or cupy. Not sure if that or an xarray-adjacent package (cupy-xarray?) is better.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660136749 https://github.com/pydata/xarray/issues/4234#issuecomment-660136749 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDEzNjc0OQ== jacobtomlinson 1610850 2020-07-17T14:26:35Z 2020-07-17T14:32:03Z CONTRIBUTOR

@jthielen Something like this?

```python import xarray as xr import cupy as cp

@xr.register_dataset_accessor("cupy") class CupyAccessor: def to_cupy(self): """Convert all data arrays to cupy."""

    for var in self.data_vars: 
        self.data_vars[var].data = cp.asarray(self.data_vars[var].data)

    return self

```

Which would then be used like this.

```python import xarray as xr import cupy as cp

ds = xr.open_mfdataset("/path/to/files/*.nc") gds = ds.cupy.to_cupy() ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660129151 https://github.com/pydata/xarray/issues/4234#issuecomment-660129151 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDEyOTE1MQ== jthielen 3460034 2020-07-17T14:12:23Z 2020-07-17T14:13:17Z CONTRIBUTOR

@jacobtomlinson Have you considered implementing an accessor for this and any other CuPy-specific functionality? This is the approach we are taking with Pint integration (pint-xarray), and I wonder if it might also be a good approach for CuPy.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613
660030480 https://github.com/pydata/xarray/issues/4234#issuecomment-660030480 https://api.github.com/repos/pydata/xarray/issues/4234 MDEyOklzc3VlQ29tbWVudDY2MDAzMDQ4MA== jacobtomlinson 1610850 2020-07-17T10:37:46Z 2020-07-17T10:37:46Z CONTRIBUTOR

cc @quasiben

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add ability to change underlying array type 659129613

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 14.806ms · About: xarray-datasette