issue_comments

Hi @DWesl, @benbovy

I got a bug very similar to what was initially report with the following minimal example (xarray version 2022.11.0) with the error:

ValueError: cannot unstack dimensions that do not have exactly one multi-index: ('z',)

It is working fine with my previous version (xarray 0.16.0) python import xarray as xr ds = xr.Dataset( data_vars=dict( mydata = ( ['across', 'along'], [[0,1],[2,3]]) ) ) stacked = ds.stack(z=("across", "along")) newlist = [None] * stacked.z.size for ii, zindex in enumerate(stacked.z.data): newlist[ii] = stacked.mydata.sel(z=zindex) newds = xr.concat(newlist, dim='z') newds['z'] = stacked.z print('xarray version:' + xr.__version__) newds.unstack(dim='z')

If I do newds = newds.reset_index('z') just before the last line, it is solved for my minimal example. But it still doesn't work for my real problem and I don't think it is the best way to proceed.

Many thanks for your help, Adrien

Hi @benbovy, my code works for 2 dimensions, but it doesn't with 1D. I think the error comes from .set_index(z=['across']) which delivers a "PandasIndex(Int64Index(" and not the expected "PandasIndex(MultiIndex(" for unstack.

I created a Pandas-MultiIndex "MultiIndex([((0,), (1,))], names=['z', 'across'])" (see code just below), but it is not exactly the same one as the one created by .stack() "PandasIndex(MultiIndex([(0,),(1,)], name='z'))" and I didn't find out how to pass the 'index' to xarray. python import pandas as pd tuples = [(0,),(1,)], index = pd.MultiIndex.from_tuples(tuples, names=["z","across"]) index

My modified minimal example is as below: python import xarray as xr ds = xr.Dataset( data_vars=dict( mydata = ( ['across', 'along'], [[0,1],[2,3]]) ), coords=dict( # <- changed across=[0,1], # <- changed along=[0,1], # <- changed ), # <- changed ) stacked = ds.isel(along=0).stack(z=["across"]) # <- changed newlist = [None] * stacked.z.size for ii, zindex in enumerate(stacked.z.data): newlist[ii] = stacked.mydata.sel(z=zindex) newds = xr.concat(newlist, dim='z') newds = newds.set_index(z=['across']) # <- changed newds.unstack(dim='z') # Fail with ValueError: cannot unstack dimensions that do not have exactly one multi-index: ('z',)

Many thanks for the explanation and the rapid answer. I am not very familiar with multi-index.

My real problem works with a DataSet with different sort of variables:

ipdb> lmmap <xarray.Dataset> Dimensions: (x_variables: 4, Observables: 2, Antenna: 4, z: 4, Ambiguities: 4, fun_variables: 8, extrema: 2) Coordinates: * x_variables (x_variables) <U3 'u' 'v' 'c_u' 'c_v' * Observables (Observables) <U6 'sigma0' 'RVL' * Antenna (Antenna) <U4 'Fore' 'MidV' 'MidH' 'Aft' fun_variables (Observables, Antenna) int64 0 1 2 3 4 5 6 7 Dimensions without coordinates: z, Ambiguities, extrema Data variables: (12/17) x (z, Ambiguities, x_variables) float64 -4.503 7.8 ... -0.4265 active_mask (z, Ambiguities, x_variables) int64 0 0 0 0 0 0 ... 0 0 0 0 0 grad (z, Ambiguities, x_variables) float64 7.816e-11 ... -1.873... fun (z, Ambiguities, Observables, Antenna) float64 0.001137 ..... jac (z, Ambiguities, fun_variables, x_variables) float64 0.359... x0 (z, Ambiguities, x_variables) float64 -7.002 10.03 ... 1.219 ... ... message (z, Ambiguities) <U42 '`xtol` termination condition is sat... success (z, Ambiguities) bool True True True True ... True True True method (z) <U3 'trf' 'trf' 'trf' 'trf' xtol (z) float64 0.001 0.001 0.001 0.001 x_scale (z, x_variables) float64 7.0 7.0 0.5 0.5 ... 7.0 7.0 0.5 0.5 bounds (z, extrema, x_variables) int64 -30 -30 -5 -5 ... 30 30 5 5

I don't know where I should apply set_index(z=["across", "along"]) I tried on the full DataSet or a single DataArray but without success. I get the following error:

ValueError: across, along variable(s) do not exist

Which is obvious as across and along variables disappeared. Thank you, I will dig in this direction why it happens.

However, it works perfectly with the minimal example. Many thanks

Thanks @benbovy, my issue is now resolved, once I have enabled the coords to follow.

I didn't see any warning for this

Many thanks for the explanation and the rapid answer. I am not very familiar with multi-index.

My real problem works with a DataSet with different sort of variables:

ipdb> lmmap <xarray.Dataset> Dimensions: (x_variables: 4, Observables: 2, Antenna: 4, z: 4, Ambiguities: 4, fun_variables: 8, extrema: 2) Coordinates: * x_variables (x_variables) <U3 'u' 'v' 'c_u' 'c_v' * Observables (Observables) <U6 'sigma0' 'RVL' * Antenna (Antenna) <U4 'Fore' 'MidV' 'MidH' 'Aft' fun_variables (Observables, Antenna) int64 0 1 2 3 4 5 6 7 Dimensions without coordinates: z, Ambiguities, extrema Data variables: (12/17) x (z, Ambiguities, x_variables) float64 -4.503 7.8 ... -0.4265 active_mask (z, Ambiguities, x_variables) int64 0 0 0 0 0 0 ... 0 0 0 0 0 grad (z, Ambiguities, x_variables) float64 7.816e-11 ... -1.873... fun (z, Ambiguities, Observables, Antenna) float64 0.001137 ..... jac (z, Ambiguities, fun_variables, x_variables) float64 0.359... x0 (z, Ambiguities, x_variables) float64 -7.002 10.03 ... 1.219 ... ... message (z, Ambiguities) <U42 '`xtol` termination condition is sat... success (z, Ambiguities) bool True True True True ... True True True method (z) <U3 'trf' 'trf' 'trf' 'trf' xtol (z) float64 0.001 0.001 0.001 0.001 x_scale (z, x_variables) float64 7.0 7.0 0.5 0.5 ... 7.0 7.0 0.5 0.5 bounds (z, extrema, x_variables) int64 -30 -30 -5 -5 ... 30 30 5 5

I don't know where I should apply set_index(z=["across", "along"]) I tried on the full DataSet or a single DataArray but without success. I get the following error:

ValueError: across, along variable(s) do not exist

Which is obvious as across and along variables disappeared. Thank you, I will dig in this direction why it happens.

However, it works perfectly with the minimal example. Many thanks

Dear all and thank you for your work on Xarray,

Link to @magau comment, I have a netcdf with multiple variables in different format (float, short, byte). Using open_mfdataset 'short' and 'byte' are converted in 'float64' (no scaling, but some masking for the float data). It doesn't raise major issue for me, but it is taking plenty of memory space for nothing.

Below an example of the 3 format from (ncdump -h): short total_nobs(time, lat, lon) ; total_nobs:long_name = "Number of SSS in the time interval" ; total_nobs:valid_min = 0s ; total_nobs:valid_max = 10000s ; float pct_var(time, lat, lon) ; pct_var:_FillValue = NaNf ; pct_var:long_name = "Percentage of SSS_variability that is expected to be not explained by the products" ; pct_var:units = "%" ; pct_var:valid_min = 0. ; pct_var:valid_max = 100. ; byte sss_qc(time, lat, lon) ; sss_qc:long_name = "Sea Surface Salinity Quality, 0=Good; 1=Bad" ; sss_qc:valid_min = 0b ; sss_qc:valid_max = 1b ;

And how they appear after opening in as xarray using open_mfdataset: total_nobs (time, lat, lon) float64 dask.array<chunksize=(48, 584, 1388), meta=np.ndarray> pct_var (time, lat, lon) float32 dask.array<chunksize=(48, 584, 1388), meta=np.ndarray> sss_qc (time, lat, lon) float64 dask.array<chunksize=(48, 584, 1388), met

Is there any recommandation? Regards

thank you for the suggestion

With your explanation, it is now clearer and it works fine. If it is possible to add your sentence explanation to the ValueError it could be good. Many thanks for your help.