issue_comments: 1062755678

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6329#issuecomment-1062755678 https://api.github.com/repos/pydata/xarray/issues/6329 1062755678 IC_kwDOAMm_X84_WF1e 6574622 2022-03-09T10:06:22Z 2022-03-09T10:06:22Z CONTRIBUTOR

Yes, that looks like the error as described in the initial post. Adding the described workaround (i.e. del buff.air.attrs["_FillValue"] in this case) leads to the next error message:

ValueError: variable 'air' already exists with different dimension sizes: {'time': 0, 'y': 250, 'x': 250} != {'time': 1, 'y': 250, 'x': 250}. to_zarr() only supports changing dimension sizes when explicitly appending, but append_dim=None.

Which is due to a mix of append-mode (mode='a') and region-write (region={'time':slice(i,i+1)}), which is e.g. out of the scope as outlined in this comment. It may or may not be possible or intended to support this, but I'm not deep enough into the design of xarray to give a definitive answer here. For me, it's unclear how this should behave. My current point of view is:

  • append: may change structure-defining metadata, must be sequential, mode='a'
  • region: may not change structure-defining metadata, can be parallel, mode='r+'

Currently, I can't really imagine how a mix of both should behave. If you can't prepare the dataset for the final shape upfront (to use region) and you also can't use append_dim, then probably what's needed is a separate method of expanding the dataset (i.e. reshape) without filling in the data. If such a thing would be available, one could (as a user) ensure that all reshaping operations are properly sequenced with region operations, but region operations could be run in parallel. (I think this is possible with plain-zarr, but I'm not aware of a corresponding xarray API).

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   1159923690