What happened?

On upgrading from xarray 2022.3.0 to 2022.6.0, f-string formatting of sparse-backed DataArray raises an exception.

What did you expect to happen?

• Code does not error, or
• A breaking change is listed in the “Breaking changes” section of the docs.

Minimal Complete Verifiable Example

```Python import pandas as pd import xarray as xr

s = pd.Series( range(4), index=pd.MultiIndex.from_product([list("ab"), list("cd")]), )

da = xr.DataArray.from_series(s, sparse=True)

print(f"{da}") ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python

xarray 2022.3.0:

<xarray.DataArray (level_0: 2, level_1: 2)> <COO: shape=(2, 2), dtype=float64, nnz=4, fill_value=nan>
Coordinates:
* level_0 (level_0) object 'a' 'b' * level_1 (level_1) object 'c' 'd'

xarray 2022.6.0:

Traceback (most recent call last):
File "/home/khaeru/bug.py", line 11, in <module> print(f"{da}") File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/common.py", line 168, in format
return self.values.format(format_spec) File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/dataarray.py", line 685, in values
return self.variable.values File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/variable.py", line 527, in values
return _as_array_or_item(self._data) File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/variable.py", line 267, in _as_array_or_item
data = np.asarray(data) File "/home/khaeru/.local/lib/python3.10/site-packages/sparse/_sparse_array.py", line 229, in array
raise RuntimeError( RuntimeError: Cannot convert a sparse array to dense automatically. To manually densify, use the todense method. ```

Anything else we need to know?

Along with the versions below, I have confirmed the error occurs with both sparse 0.12 and sparse 0.13.

Environment

This adds some code (from iiasa/ixmp#317) that allows DataArray.to_series() to be called without invoking sparse.COO.todense() when that is the backing data type.

I'm aware this needs some improvement to meet the standard of the existing codebase, so I hope I could ask for some guidance on how to address the following points (including whom to ask about them): - [ ] Make the same improvement in {DataArray,Dataset}.to_dataframe(). - [ ] Possibly move the code out of dataarray.py to a more appropriate location (where?). - [ ] Possibly check for sparse.COO explicitly instead of xarray.core.pycompat.sparse_array_type. Other SparseArray subclasses, e.g. DOK, may not have the same attributes.

Standard items: - [ ] Tests added. - [x] Passes isort -rc . && black . && mypy . && flake8 (Sort of: these wanted to modify 7 files beyond the one I touched; didn't commit these changes.) - [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API.

MCVE Code Sample

```python import numpy as np import pandas as pd import sparse import xarray as xr

Indices and raw data

foo = [f'foo{i}' for i in range(6)] bar = [f'bar{i}' for i in range(6)] raw = np.random.rand(len(foo) // 2, len(bar))

DataArray

a = xr.DataArray( data=sparse.COO.from_numpy(raw), coords=[foo[:3], bar], dims=['foo', 'bar'])

print(a.data.fill_value) # 0.0

Created from a pd.Series

b_series = pd.DataFrame(raw, index=foo[3:], columns=bar) \ .stack() \ .rename_axis(index=['foo', 'bar']) b = xr.DataArray.from_series(b_series, sparse=True)

print(b.data.fill_value) # nan

Works despite inconsistent fill-values

a + b a * b

Fails: complains about inconsistent fill-values

xr.concat([a, b], dim='foo') # ***

The fill_value argument doesn't help

xr.concat([a, b], dim='foo', fill_value=np.nan)

def fill_value(da): """Try to coerce one argument to a consistent fill-value.""" return xr.DataArray( data=sparse.as_coo(da.data, fill_value=np.nan), coords=da.coords, dims=da.dims, name=da.name, attrs=da.attrs, )

Fails: "Cannot provide a fill-value in combination with something that

already has a fill-value"

print(xr.concat([a.pipe(fill_value), b], dim='foo'))

If we cheat by recreating 'a' from scratch, copying the fill value of the

intended other argument, it works again:

a = xr.DataArray( data=sparse.COO.from_numpy(raw, fill_value=b.data.fill_value), coords=[foo[:3], bar], dims=['foo', 'bar']) c = xr.concat([a, b], dim='foo')

print(c.data.fill_value) # nan

But simple operations again create objects with potentially incompatible

fill-values

d = c.sum(dim='bar') print(d.data.fill_value) # 0.0 ```

Expected

concat() can be used without having to create new objects; i.e. the line marked *** just works.

Problem Description

Some basic xarray manipulations don't work on sparse.COO-backed objects.

xarray should automatically coerce objects into a compatible state, or at least provide users with methods to do so. Behaviour should also be documented, e.g. in this instance, which operations (here, .sum()) modify the underlying storage format in ways that necessitate some kind of (re-)conversion.

Output of xr.show_versions()

With xarray 0.7.2, following this basic example from the docs, but with a modification in the last line to use pd.Period instead of pd.Timestamp:

``` python import numpy as np import xarray as xr

temp = 15 + 8 * np.random.randn(2, 2, 3) precip = 10 * np.random.rand(2, 2, 3) lon = [[-99.83, -99.32], [-99.79, -99.23]] lat = [[42.25, 42.21], [42.63, 42.59]]

ds = xr.Dataset({'temperature': (['x', 'y', 'time'], temp), 'precipitation': (['x', 'y', 'time'], precip)}, coords={'lon': (['x', 'y'], lon), 'lat': (['x', 'y'], lat), 'time': pd.date_range('2014-09-06', periods=3), 'reference_time': pd.Period('2014')}) ```

This raises:

ValueError: dimensions ('reference_time',) must have the same length as the number of data dimensions, ndim=0

I noticed (#645) that there are other issues stemming from pandas' PeriodIndex & company, so if this is not a straightforward fix I will understand!

I am working on some code that creates xray.Datasets with a 'bool' dtype.

Trying to call Dataset.to_netcdf() on this code causes _nc4_values_and_dtype() to raise a ValueError, so I added these few lines to force the storage of these variables as 1-byte integers.

Perhaps it should be 'u1' instead; I can change that if need be.