issue_comments

@Material-Scientist We have decent support for pydata/sparse arrays. It seems like these would work for you

We do not support the pandas extension arrays at the moment.

I understand why this could be useful, but I don't see how we could possibly make it work.

The notion of "fixed dimension size" is fundamental to both NumPy arrays (upon which xarray is based) and the xarray Dataset/DataArray. There are lots of various workarounds (e.g., using padding or a MultiIndex) but first class support for jagged arrays would break our existing data model too severely.

I have a similar use case and I often use MultiIndex, which (partly) enables to handle hierarchical data structure.

For example, ```python In [1]: import xarray as xr ...: import numpy as np ...: ...: # image 0, size [3, 4] ...: data0 = xr.DataArray(np.arange(12).reshape(3, 4), dims=['x', 'y'], ...: coords={'x': np.linspace(0, 1, 3), ...: 'y': np.linspace(0, 1, 4), ...: 'image_index': 0}) ...: # image 1, size [4, 5] ...: data1 = xr.DataArray(np.arange(20).reshape(4, 5), dims=['x', 'y'], ...: coords={'x': np.linspace(0, 1, 4), ...: 'y': np.linspace(0, 1, 5), ...: 'image_index': 1}) ...: ...: data = xr.concat([data0.expand_dims('image_index').stack(xy=['x', 'y', 'image_index']), ...: data1.expand_dims('image_index').stack(xy=['x', 'y', 'image_index'])], ...: dim='xy')

In [2]: data Out[2]: <xarray.DataArray (xy: 32)> array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]) Coordinates: * xy (xy) MultiIndex - x (xy) float64 0.0 0.0 0.0 0.0 0.5 0.5 0.5 0.5 1.0 1.0 1.0 ... - y (xy) float64 0.0 0.3333 0.6667 1.0 0.0 0.3333 0.6667 1.0 ... - image_index (xy) int64 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 ...

In [3]: data.sel(image_index=0) # gives data0 Out[3]: <xarray.DataArray (xy: 12)> array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]) Coordinates: * xy (xy) MultiIndex - x (xy) float64 0.0 0.0 0.0 0.0 0.5 0.5 0.5 0.5 1.0 1.0 1.0 1.0 - y (xy) float64 0.0 0.3333 0.6667 1.0 0.0 0.3333 0.6667 1.0 0.0 ...

In [4]: data.sel(x=0.0) # x==0.0 for both images Out[4]: <xarray.DataArray (xy: 9)> array([0, 1, 2, 3, 0, 1, 2, 3, 4]) Coordinates: * xy (xy) MultiIndex - y (xy) float64 0.0 0.3333 0.6667 1.0 0.0 0.25 0.5 0.75 1.0 - image_index (xy) int64 0 0 0 0 1 1 1 1 1 ```

<s>I think the above solution is essentially equivalent with

all images of the same size into its own dimension</s>

EDIT: I didn't understand the comment correctly. The above corresponds to that all the images are flattened out and combined along one large dimension.

"Supported", yes, in the sense that you can create a DataArray for each of your differently sized arrays without any problem. If you want to store them all in a Dataset, you'll have to give a different dimension name for each new dimension, which can be clumsy.

However, it is true that xarray shines at handling more structured data and that most examples in the docs are those of dataset variables sharing similar dimensions. What kind of "support" exactly were you thinking of?