issue_comments: 636849900

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/4113#issuecomment-636849900 https://api.github.com/repos/pydata/xarray/issues/4113 636849900 MDEyOklzc3VlQ29tbWVudDYzNjg0OTkwMA== 36678697 2020-06-01T13:06:02Z 2020-06-01T13:06:02Z NONE

I think it depends on the chunk size.

Yes, I'm not very familiar with chunks, it seems that it's not good to have too many of them.

I am not sure where 512 comes from in your example (maybe dask does something).

Sorry it should have been (100, 2048), it comes from the second dimension of stacking (explained below). My screenshot was for .stack(px=("y", "x")), my bad.

If I work with chunks=dict(x=128, y=128), the chunksize after the stacking was (100, 16384), which is reasonable (z=100, px=(128, 128)).

Yes, after some more experiments I found out that the second chunksize after stacking is (100, X) where X is a multiple of the size of the second stacking dimension (here "y"), hence why it is working in your case (128 * 128 == 2048 * 8).

The formula for X is something like:

shape[1] * ( (x_chunk * y_chunk) // shape[1] + bool((x_chunk * y_chunk) % shape[1]) )

So, minimum value for X is shape[1] (size of "y" dim, hence my case with small values for x_chunk and y_chunk).

That's why I was saying that "chunks along the second stacking dimension seem to be merged". This might be normal, just unexpected, and still quite obscure for me.

And it must be happening on dask side anyway. Thanks a lot for your insights.

You can do reset_index before saving it into the netCDF, but it requires another computation when creating the MultiIndex after loading.

Ah yes, thanks! I thought reset_index was similar to unstack for indexes created with stack.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   627735640