html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/4234#issuecomment-739094107,https://api.github.com/repos/pydata/xarray/issues/4234,739094107,MDEyOklzc3VlQ29tbWVudDczOTA5NDEwNw==,2448579,2020-12-05T00:48:49Z,2020-12-05T00:48:49Z,MEMBER,"The indexes story will change soon, we may even have our own index classes.

We should have pretty decent support for NEP-18 arrays in `DataArray.data` though, so IMO that's the best thing to try out and see where the issues remain. 

NEP35 is cool; looks like we should use it in our `*_like` functions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-739066140,https://api.github.com/repos/pydata/xarray/issues/4234,739066140,MDEyOklzc3VlQ29tbWVudDczOTA2NjE0MA==,1403768,2020-12-04T22:56:23Z,2020-12-04T22:56:23Z,MEMBER,"Wanted to update on some recent work.  With NEP35 (https://github.com/numpy/numpy/pull/16935, https://github.com/numpy/numpy/pull/17787) experimentally in NumPy, I've been exploring a bit more on using GPUs with xarray starting off with basic groupby workflows.  There are places in the code where xarray calls pandas directly.  For example, when building Indexes:

https://github.com/pydata/xarray/blob/7152b41fa80a56db0ce88b241fbe4092473cfcf0/xarray/core/dataset.py#L150-L153

This is going to be challenging primarily because xarray will need to determine which DataFrame library to use during Index creation (possibly other DataFrame objects).  While there is a [consortium](https://data-apis.org/) developing potential solutions for libraries to leverage multiple dataframe libraries I was going to keep hacking away and see what other issues may be lurking. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660192634,https://api.github.com/repos/pydata/xarray/issues/4234,660192634,MDEyOklzc3VlQ29tbWVudDY2MDE5MjYzNA==,1610850,2020-07-17T16:07:31Z,2020-07-17T16:08:34Z,CONTRIBUTOR,"Those `as_` methods sounds good. Would `as_dense` be the same as `as_numpy`?

Yeah it is possible to read direct to GPU from storage with [GDS](https://developer.nvidia.com/blog/gpudirect-storage/). We've experimented a little with zarr, I expect if something like zarr got GDS support and a zarr dataset was configured to use GDS then `xr.open_zarr('zarr/path')` would be already backed by cupy because the zarr internal array would be cupy.

> Re: index variables.Can we avoid this for now?

I think that would be best. 

However I did run into issues when trying to run the [Compare weighted and unweighted mean temperature](http://xarray.pydata.org/en/stable/examples/area_weighted_temperature.html) example with cupy. In that example the `weights` data array is generated from the latitude index. So the resulting `DataArray` is backed by numpy. I would expect that if it were a cuDF Index it would end up as a cupy data array.

In my testing I just cast the `weights` data array to cupy and carried on. So perhaps with this change users will just need to sprinkle some `weights = weights.as_cupy()` or `weights = weights.asarray(cp.ndarray)` type calls throughout their code when they need to.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660180546,https://api.github.com/repos/pydata/xarray/issues/4234,660180546,MDEyOklzc3VlQ29tbWVudDY2MDE4MDU0Ng==,2448579,2020-07-17T15:46:15Z,2020-07-17T15:46:15Z,MEMBER,"See similar discussion for sparse here: https://github.com/pydata/xarray/issues/3245

`asarray` makes sense to me.

I think we are also open to special `as_sparse`, `as_dense`, `as_cupy` that return xarray objects with converted arrays.

A `to_numpy_data` method (or `as_numpy`?) would always coerce to numpy appropriately.

IIRC there's some way to read from disk to GPU, isn't there? So it makes sense to expose that in our `open_*` functions.

Re: index variables.Can we avoid this for now? Or are there going to be performance issues? The general problem will be handled as part of the index refactor (we've deferred pint support for indexes for this reason).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660156875,https://api.github.com/repos/pydata/xarray/issues/4234,660156875,MDEyOklzc3VlQ29tbWVudDY2MDE1Njg3NQ==,3460034,2020-07-17T15:03:01Z,2020-07-17T15:03:01Z,CONTRIBUTOR,"The accessors could also be where wrapped xarray methods could go (so in your previous example, it could be `time_slice.cupy.plot()` to handle the NumPy conversion internally), but based on your explanations I'd agree that most of this makes more sense to be compatibility code that goes in xarray itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660154476,https://api.github.com/repos/pydata/xarray/issues/4234,660154476,MDEyOklzc3VlQ29tbWVudDY2MDE1NDQ3Ng==,1610850,2020-07-17T14:58:53Z,2020-07-17T14:58:53Z,CONTRIBUTOR,"The only things I can think of that would make sense initially in an accessor would be `to_cupy`/`from_cupy` to move back and forth from the GPU, plus maybe an alias for `get` to `from_cupy` to match the cupy API.

An accessor does seem like a reasonable place to put that logic, but it also seems like a tiny amount of code to make, ship and maintain a separate package for. Plus those methods will either need to be used or duplicated in the core codebase to support things like plotting.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660148433,https://api.github.com/repos/pydata/xarray/issues/4234,660148433,MDEyOklzc3VlQ29tbWVudDY2MDE0ODQzMw==,1610850,2020-07-17T14:48:01Z,2020-07-17T14:54:41Z,CONTRIBUTOR,"This does sound like an option. However there are many situations within xarray where we need explicit cupy logic. Converting back to numpy before plotting is one example. I don't think that kind of logic can live in an accessor.

Unless you expect users to do something like this.

```python
import xarray as xr
import cupy as cp

ds = xr.tutorial.load_dataset(""air_temperature"")
gds = ds.cupy.to_cupy()

# Do some manipulation on the GPU

# Grab a time slice
time_slice = gds.air.isel(time=0)

time_slice.cupy.to_numpy().plot()  # I would hope that time_slice.plot() would work
```

I would be tempted to say that cupy is more like dask in that it is trying to implement the numpy array interface exactly but in a different paradigm (distributed, GPU, etc). And of course there are limitations and divergences because of the different paradigm. However it's not the same as pint which is trying to extend numpy and add more functionality.

So it makes sense to me that accessors for pint exist to add this extra functionality to xarray. But at least in theory cupy should be a drop-in replacement for numpy. So I don't expect a huge amount of logic will live in an accessor, compared to the amount of compatibility code that will need to go into xarray itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660148938,https://api.github.com/repos/pydata/xarray/issues/4234,660148938,MDEyOklzc3VlQ29tbWVudDY2MDE0ODkzOA==,3460034,2020-07-17T14:48:59Z,2020-07-17T14:48:59Z,CONTRIBUTOR,@jacobtomlinson Makes complete sense! Just wanted to make sure the option was considered as a possibility.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660142143,https://api.github.com/repos/pydata/xarray/issues/4234,660142143,MDEyOklzc3VlQ29tbWVudDY2MDE0MjE0Mw==,3460034,2020-07-17T14:36:40Z,2020-07-17T14:36:40Z,CONTRIBUTOR,"@jacobtomlinson Indeed! A similar `to_cupy` could also be made for existing DataArrays on a DataArray accessor, and any other CuPy-related attributes/methods could be attached to the same accessors as well.

Though, to have those accessors be registered with just importing `xarray` and `cupy`, they would of course need to live in either `xarray` or `cupy`. Not sure if that or an xarray-adjacent package (`cupy-xarray`?) is better.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660136749,https://api.github.com/repos/pydata/xarray/issues/4234,660136749,MDEyOklzc3VlQ29tbWVudDY2MDEzNjc0OQ==,1610850,2020-07-17T14:26:35Z,2020-07-17T14:32:03Z,CONTRIBUTOR,"@jthielen Something like this?

```python
import xarray as xr
import cupy as cp


@xr.register_dataset_accessor(""cupy"")
class CupyAccessor:
    def to_cupy(self):
        """"""Convert all data arrays to cupy.""""""

        for var in self.data_vars: 
            self.data_vars[var].data = cp.asarray(self.data_vars[var].data)

        return self

```

Which would then be used like this.

```python
import xarray as xr
import cupy as cp

ds = xr.open_mfdataset(""/path/to/files/*.nc"")
gds = ds.cupy.to_cupy()
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660129151,https://api.github.com/repos/pydata/xarray/issues/4234,660129151,MDEyOklzc3VlQ29tbWVudDY2MDEyOTE1MQ==,3460034,2020-07-17T14:12:23Z,2020-07-17T14:13:17Z,CONTRIBUTOR,"@jacobtomlinson Have you considered [implementing an accessor](http://xarray.pydata.org/en/stable/internals.html#extending-xarray) for this and any other CuPy-specific functionality? This is the approach we are taking with Pint integration ([pint-xarray](https://github.com/xarray-contrib/pint-xarray/)), and I wonder if it might also be a good approach for CuPy.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613
https://github.com/pydata/xarray/issues/4234#issuecomment-660030480,https://api.github.com/repos/pydata/xarray/issues/4234,660030480,MDEyOklzc3VlQ29tbWVudDY2MDAzMDQ4MA==,1610850,2020-07-17T10:37:46Z,2020-07-17T10:37:46Z,CONTRIBUTOR,cc @quasiben,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,659129613