This PR adds to_iris and from_iris methods to DataSet. I've added this because I frequently find myself writing little list and dictionary comprehensions to pack and unpack both DataSets from DataArrays and Iris CubeLists from Cubes.

• [x] Tests added (for all bug fixes or enhancements)
• [ ] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Is your feature request related to a problem? Please describe.

In order to use Xarray with alternative array types like cupy the user needs to be able to specify the underlying array type without digging into internals.

Right now I'm doing something like this.

```python import xarray as xr import cupy as cp

ds = xr.tutorial.load_dataset("air_temperature") ds.air.data = cp.asarray(ds.air.data) ```

However this will become burdensome when there are many data arrays and feels brittle and prone to errors.

As I see it a conversion could instead be done in a couple of places; on load, or as a utility method.

Currently Xarray supports NumPy and Dask array well. Numpy is the defrault and the way you specify whether a Dask array should be used is to give the chunks kwargs to an open_ function or by calling .chunk() on a DataSet or DataArray.

Side note: There are a few places where the Dask array API bleeds into Xarray in order to have compatibility, the chunk kwarg/method is one, the .compute() method is another. I'm hesitant to do this for other array types, however surfacing the cupy.ndarray.get method could feel natural for cupy users. But for now I think it would be best to take Dask as a special case and try and be generic for everything else.

Describe the solution you'd like

For other array types I would like to propose the addition of an asarray kwarg for the open_ methods and an .asarray() method on DataSet and DataArray. This should take either the array type cupy.ndarray, the asarray method cp.asarray, or preferably either.

This would result in something like the following.

```python import xarray as xr import cupy as cp

ds = xr.open_mfdataset("/path/to/files/*.nc", asarray=cp.ndarray)

or

ds = xr.open_mfdataset("/path/to/files/*.nc") gds = ds.asarray(cp.ndarray) ```

These operations would convert all data arrays to cupy arrays. For the case that ds is backed by Dask arrays it would use map_blocks to cast each block to the appropriate array type.

It is still unclear what to do about index variables, which are currently of type pandas.Index. For cupy it may be more appropriate to use a cudf.Index instead to ensure both are on the GPU. However this would add a dependency on cudf and potentially increase complexity here.

Describe alternatives you've considered

Instead of an asarray kwarg/method something like to_cupy/from_cupy could be done. However I feel this makes less sense because the object type is not changing, just that of the underlying data structure.

Another option would be to go more high level with it. For example a gpu kwarg and to_gpu/from_gpu method could be added in the same way. This would abstract things even further and give users a choice about hardware rather than software. This would also be a fine solution but I think it may special case too much and a more generic solution would be better.

Additional context Related to #4212.

I'm keen to start implementing this. But would like some discussion/feedback before I dive in here.

I'm intending on working on cupy support in xarray along with @quasiben. Thanks for the warm welcome in the xarray dev meeting yesterday!

I'd like to use this issue to track cupy support and discuss certain design decisions. I appreciate there are also issues such as #4208, #3484 and #3232 which are related to cupy support, but maybe this could represent an umbrella issue for cupy specifically.

The main goal here is to improve support for array types other than numpy and dask in general. However, it is likely there will need to be some cupy specific compatibility code in xarray. (@andersy005 raised issues with calling __array__ on cupy in #3232 for example).

I would love to hear from folks wanting to use cupy with xarray to help build up some use cases for us to develop against. We have some ideas but more are welcome.

My first steps here will be to add some tests which use cupy. These will skip in the main CI but we will also look at running xarray tests on some GPU CI too as we develop. A few limited experiments that I've run seem to work, so I'll start with tests which reproduce those.

This implements solution 2 for #4231.

cc @quasiben

• [x] Closes #4231
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8

Related to #4212

When trying to get the Calculating Seasonal Averages from Timeseries of Monthly Means example from the documentation to work with cupy I'm experiencing an unexpected Unsupported type <class 'numpy.ndarray'> error when calling ds_unweighted = ds.groupby('time.season').mean('time')

I dug through this with @quasiben and it seems to be related to the as_shared_dtype function.

What happened:

Running the MCVE below results in Unsupported type <class 'numpy.ndarray'>. It seems at somewhere in the stack there is a call to _replace_nan(a, 0) where the cupy array is having nan values replaced with 0. This ends up as a call to xarray.core.duck_array_ops.where with the "is nan", 0 and the cupy array being passed.

However _where calls as_shared_dtype on the 0 and cupy array, which converts the 0 to a scalar numpy array.

Cupy is then passed this numpy array to it's where function which does raises the exception.

What you expected to happen:

The cupy.where function can either take a Python int/float or a cupy array, not a numpy scalar.

Therefore a few things could be done here: 1. Xarray could not convert the int/float to a numpy array 1. It could convert it to a cupy array 1. Cupy could be modified to accept a numpy scalar.

We thew together a quick fix for option 2, which I'll put in a draft PR. But happy to discuss the alternatives.

Minimal Complete Verifiable Example:

```python import numpy as np import pandas as pd import xarray as xr import matplotlib.pyplot as plt

import cupy as cp

Load data

ds = xr.tutorial.open_dataset("rasm").load()

Move data to GPU

ds.Tair.data = cp.asarray(ds.Tair.data)

ds_unweighted = ds.groupby("time.season").mean("time")

Calculate the weights by grouping by 'time.season'.

month_length = ds.time.dt.days_in_month weights = ( month_length.groupby("time.season") / month_length.groupby("time.season").sum() )

Test that the sum of the weights for each season is 1.0

np.testing.assert_allclose(weights.groupby("time.season").sum().values, np.ones(4))

Move weights to GPU

weights.data = cp.asarray(weights.data)

Calculate the weighted average

ds_weighted = ds * weights ds_weighted = ds_weighted.groupby("time.season") ds_weighted = ds_weighted.sum(dim="time") ```

Anything else we need to know?:

Environment:

Added some initial unit tests for cupy. Mainly to create a place for cupy tests to go and to check some basic functionality.

I've created a fixture which constructs the dataset from the Toy weather data example and converts the underlying arrays to cupy. Then I've added a test which checks that after calling operations such as mean and groupby the resulting DataArray is still backed by a cupy array.

The main penalty with working on GPUs is accidentally shunting data back and forth between the GPU and system memory. Copying data over the PCI bus is slow compared to the rest of the work so should be avoided. So this first test is checking that we are leaving things on the GPU.

Because this data copying is so expensive cupy have intentionally broken the __array__ method and introduced a .get method instead. This means that users have to be explicit in converting back to numpy and copying back to the main memory. Therefore we will need to add some logic to xarray to use .get in appropriate situations such as plotting.

• [x] Releated to #4212
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8

Problem description

Using xarray 0.10.7 I'm getting the following error when trying to write out a zarr.

ValueError: Zarr requires uniform chunk sizes excpet for final chunk. Variable ((3, 3, 3, 3), (3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2), (1, 1, 1), (100, 100, 100, 100, 100, 100), (100, 100, 100, 100, 100, 100, 100, 100)) has incompatible chunks. Consider rechunking using `chunk()`.

Those chunks look fine to me, only one has an inconsistent chunking and it's the final chunk in the second index. Seems related to #2225.