issue_comments

Thank you,

Next time I will triple check and exclude those variables from being expanded in dimension.

Thank you for your time.

Dear Slevang,

Thank you very much for your reply, I was indeed trying the same without the wshedOutvariable. Deleting this, the problem that the dataset increases too much seems to be of less impact, indeed I can use the xr.where on larger dataset, however:

(a.nbytes - da_fam_bulk_noWshed.nbytes)/1000000 37.87776 MB

The two datasets (a is after the xr.where of the da_fam_bulk_noWshed dataset) without this variable differ by about 37MB, being a bigger than the original. This small increment for me is important due to the fact that I have more than 1000 files.

There is a solution?

Thank you a lot,

Thank you so much for your very quick reply,

The files are .nc files (netCDF), generated with xarray, Here there is the Panoply screenshot:

This is the display(a)

I double-checked the data and they seem to be float64.

As you said, they do not change dtype and using only a variable, this is the result: da_fam_bulk['tp'].nbytes 41665536 xr.where(da_fam_bulk['tp'] != 0,da_fam_bulk['tp'],np.nan).nbytes 41665536

So using only one variable the problem disappears.

dm 41665536 xr.where 41665536 tp 41665536 xr.where 41665536 gamma_best 41665536 xr.where 41665536 m0 41665536 xr.where 41665536

I checked all the variables, the problem exists only when using the whole dataset.

Do you have any suggestion?

Thank you,

Thank you very much for your fast reply,

repr(da_fam_bulk)

<xarray.Dataset> Dimensions: (fami: 11, site: 1233, freq: 32, dir: 24, time: 384) Coordinates: * fami (fami) int64 1 2 3 4 5 6 7 8 9 10 11 * site (site) int64 51 54 72 75 90 93 ... 7004 7006 7049 7052 7094 7128 lat (site) float32 ... lon (site) float64 ... * freq (freq) float64 0.0373 0.04103 0.04513 ... 0.5917 0.6509 0.7159 * dir (dir) float64 0.0 15.0 30.0 45.0 ... 300.0 315.0 330.0 345.0 * time (time) datetime64[ns] 1989-01-01 1989-02-01 ... 2020-12-01 Data variables: dm (fami, time, site) float64 ... tp (fami, time, site) float64 ... gamma_best (fami, time, site) float64 ... m0 (fami, time, site) float64 0.04069 0.0 0.04612 ... 0.0 0.0 0.0 tm02 (fami, time, site) float64 ... hs (fami, time, site) float64 0.8068 0.0 0.8591 0.0 ... 0.0 0.0 0.0 SI (fami, time, site) float64 ... dp (fami, time, site) float64 ... m0tot (time, site) float64 0.04069 0.004237 0.04612 ... 0.1219 0.08013 m0_m0tot (fami, time, site) float64 1.0 0.0 1.0 0.0 ... 0.0 0.0 0.0 0.0 wshedOut (freq, dir, site) float64 ... I am not using a notebook, however, I paste here the screenshot of the notebook:

Thank you,