issue_comments

Hello @liuyenting! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file xarray/backends/imageio.py:

Line 22:48: W504 line break after binary operator Line 136:13: E265 block comment should start with '# '

OK, maybe this would indeed be an appropriate approach if you want to support writing files with our current interface. Unfortunately, our current interface is indeed very inadequate for such use-cases -- as you have no-doubt noticed! This is on our soon to be published roadmap to fix, but it will be a longer process.

Generally, image containers opened by imageio represents a single Dataset only, which is why I default names to their readout sequences. However, there are indeed cases where multiple variables exist, such as pyramids of image sequences, each pyramid layer represents a different resolution for the sequence.

Can you show some examples of how imageio works: - in the case of a single array - in the case of multiple variables ?

However, it is my understanding, and please correct me if I am wrong, that rasterio backend is aimed at being a read-only I/O for xarray without having the extensibility for write ability, and I'm really eager to keep the ability to aggregate multiple files through dask, as well as saving data through imageio transparently (if plausible) :p

This can be done regardless. You can add open_imageio and to_imageio methods to DataArrays/Datasets without the existing abstractions which were written for NetCDF-like files.

Do you already have an idea of how the to_imageio API would look like? This looks quite a complex topic...

if xarray's rasterio backend wouldn't be a better template for an imageio backend instead.

I started with rasterio in the beginning, since TIFF-like containers are not modifiable, cache is crucial. However, it is my understanding, and please correct me if I am wrong, that rasterio backend is aimed at being a read-only I/O for xarray without having the extensibility for write ability, and I'm really eager to keep the ability to aggregate multiple files through dask, as well as saving data through imageio transparently (if plausible) :p

On second thought, maybe open_rasterio type of approach with to_zarr for saving is more preferable? But what are the possible approaches for open_mfdataset (and the potential of using dask for out-of-memory file I/O)?

For example, can imageio open any file which resembles a Dataset (i.e.: more than one variable with different datatypes), or would a DataArray be enough?

Generally, image containers opened by imageio represents a single Dataset only, which is why I default names to their readout sequences. However, there are indeed cases where multiple variables exist, such as pyramids of image sequences, each pyramid layer represents a different resolution for the sequence.

I must admit that a single DataArray is more applicable to the general usage.

Yes, I would recommend taking a look at open_rasterio() for an example of reading image files. The other backend examples map much more closely to the netCDF data model.

Thanks! Without looking into too much detail (yet), I think it would be a great addition!

Correct me if I'm wrong, but I wonder if xarray's rasterio backend wouldn't be a better template for an imageio backend instead. For example, can imageio open any file which resembles a Dataset (i.e.: more than one variable with different datatypes), or would a DataArray be enough?