import numpy as np import xarray as xr

Creates DataArrays

nt = 4 time = np.arange (nt) * 86400.0 time = xr.DataArray (time, coords=[time,], dims=["time",]) aa = time * 2.0

Adding attributes to DataArrays

time.attrs['units'] = "second" aa.attrs['units'] = "whatever"

Attributes are visible in the DataArrays

print ('----------> time DataArray: ') print (time) print ('----------> aa DataArray : ' ) print (aa) print ('----------> aa attributes : ') print (aa.attrs )

Creating a Dataset

ds = xr.Dataset( { "aa": (["time",], aa), }, coords={"time": (["time",], time), }, )

Attributes are not visible in the Dataset

print ('----------> DataSet before setting attributes') print (ds)

My request #1 : attributes of the DataArrays should be added to the DataSet (may be optional)

print ('----------> Attributes of aa in DataSet : none') print ( ds['aa'].attrs ) print ('----------> Attributes of aa outside DataSet : still here') print ( aa.attrs )

print ('----------> Attributes are not written to the NetCDF file') ds.to_netcdf ('sample1.nc')

Adding attributes directly to the Dataset

Attributes are still not visible in the Dataset

print ('----------> DataSet after setting attributes : attributes not shown' ) ds=ds.assign_attrs({'Visible':'NotInvisibleMan'}) ds['time'].attrs['units']="second" ds['aa'].attrs['units']="whatever"
ds.to_netcdf('safeReturn.nc') print(xr.open_dataset('safeReturn.nc').attrs) print(xr.open_dataset('safeReturn.nc')['aa'].attrs) print(xr.open_dataset('safeReturn.nc')['time'].attrs)

I'm sorry I couldn't be more helpful this time

It works again, i have tried to save a netcdf file what produces the bug, but nothing. The traceback was from debug mode of django.

Linux streamDebian 5.10.0-6-amd64 #1 SMP Debian 5.10.28-1 (2021-04-09) x86_64 GNU/Linux

xr.show_versions()

INSTALLED VERSIONS

commit: None python: 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] python-bits: 64 OS: Linux OS-release: 5.10.0-6-amd64 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.17.0 pandas: 1.2.4 numpy: 1.19.5 scipy: 1.6.3 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.04.1 distributed: None matplotlib: 3.4.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 52.0.0 pip: 20.3.4 conda: None pytest: 6.0.2 IPython: 7.20.0 sphinx: None

Thanks for your quick response, it is true that you have greatly improved the examples in the documentation, I do not know to what extent it would solve any lack. The np.ufunc examples seemed insufficient a year ago but you have already solved. And it is always better to be aware of the improvements that you are introducing before writing new documentation. Thanks for your attention.

Shall i close this issue?

Ok, thanks @max-sixty and @keewis . I hope it is useful for the proper and efficient use of xarray. you know better what development path it is taking.

Hi again @max-sixty , thanks for your advice, sure, the outputs have a lot of sense. I think this new example has the good structure like another examples. https://github.com/JavierRuano/ASI_Steady/blob/main/Examples/AirStagnationIndex_Wang_Xarray_Example.ipynb

Regards Javier Ruano.

I think it is interesting the operations over the time axis with numpy reduce, and the ufuncs operations. From my point of view there are a lot of pandas and dask users who could learn to use xarray with that example or what they are using netcdf-4 and numpy directly. I have created the example from the library, and it is not problem to change it. But it shows a stagnation calculation, it was the intention.

But i understand what xarray is climate focused and you know if the topic is interesting and the example is really useful for your project. For me xarray have been very useful.

If you prefer another type of example, we could refactor it. Regards Javier Ruano.

I use http://xarray.pydata.org/en/stable/generated/xarray.apply_ufunc.html because it is faster.

El mié., 20 may. 2020 a las 20:01, Javier Ruano (javier.ruanno@gmail.com) escribió:

if you append compute() it should not be a lazy operation. But my advice is like user only.

El mié., 20 may. 2020 a las 19:51, Rob Hetland (notifications@github.com) escribió:

I have a large DataSet, including these DataArrays:

<xarray.DataArray 'temp' (ocean_time: 1325, s_rho: 30, eta_rho: 602, xi_rho: 677)> dask.array<concatenate, shape=(1325, 30, 602, 677), dtype=float32, chunksize=(1, 1, 602, 677)>

and

<xarray.DataArray 'zeta' (ocean_time: 1325, eta_rho: 602, xi_rho: 677)> dask.array<concatenate, shape=(1325, 602, 677), dtype=float32, chunksize=(1, 602, 677)>

(The coordinates and attributes excluded for brevity, but they match in the right ways.)

When I do math operations with the 4D DataArray (temp) and 3D DataArray (zeta), no problem:

ds.zeta * ds.temp

<xarray.DataArray (ocean_time: 1325, eta_rho: 602, xi_rho: 677, s_rho: 30)> dask.array<mul, shape=(1325, 602, 677, 30), dtype=float32, chunksize=(1, 602, 677, 1)>

This returns an object instantly, and the result is lazily evaluated. However, if I just try to add temp to itself,

ds.temp + ds.temp

this fails (eventually) as my medium sized computer runs out of memory, since it starts to evaluate the numbers as if I did a compute() or asked for the values. Note 2ds.temp or ds.temp*2 is lazily evaluated, and returns an object instantly. Chunk size does not seem to be an issue, as I have tried a number of reasonable choices without success.

Why can't such simple math operations between two large arrays also be lazily evaluated?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4085, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO2ZUJJOVNCN4JRJYDTRSQKBTANCNFSM4NGFVQRQ .

if you append compute() it should not be a lazy operation. But my advice is like user only.

El mié., 20 may. 2020 a las 19:51, Rob Hetland (notifications@github.com) escribió:

I have a large DataSet, including these DataArrays:

<xarray.DataArray 'temp' (ocean_time: 1325, s_rho: 30, eta_rho: 602, xi_rho: 677)> dask.array<concatenate, shape=(1325, 30, 602, 677), dtype=float32, chunksize=(1, 1, 602, 677)>

and

<xarray.DataArray 'zeta' (ocean_time: 1325, eta_rho: 602, xi_rho: 677)> dask.array<concatenate, shape=(1325, 602, 677), dtype=float32, chunksize=(1, 602, 677)>

(The coordinates and attributes excluded for brevity, but they match in the right ways.)

When I do math operations with the 4D DataArray (temp) and 3D DataArray (zeta), no problem:

ds.zeta * ds.temp

<xarray.DataArray (ocean_time: 1325, eta_rho: 602, xi_rho: 677, s_rho: 30)> dask.array<mul, shape=(1325, 602, 677, 30), dtype=float32, chunksize=(1, 602, 677, 1)>

This returns an object instantly, and the result is lazily evaluated. However, if I just try to add temp to itself,

ds.temp + ds.temp

this fails (eventually) as my medium sized computer runs out of memory, since it starts to evaluate the numbers as if I did a compute() or asked for the values. Note 2ds.temp or ds.temp*2 is lazily evaluated, and returns an object instantly. Chunk size does not seem to be an issue, as I have tried a number of reasonable choices without success.

Why can't such simple math operations between two large arrays also be lazily evaluated?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4085, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO2ZUJJOVNCN4JRJYDTRSQKBTANCNFSM4NGFVQRQ .

I think another solution you could add an aleat microsecond or nanosecond field ... in the datetime index and the index should be different. You could test if there are colision in the index (it means the index<->time is not empty like the sort algorithm, something similar to https://en.wikipedia.org/wiki/Radix_sort)

It is like the reverse of this solution

https://stackoverflow.com/questions/32827169/python-reduce-precision-pandas-timestamp-dataframe/32827432

Add another coordinate could be overloaded and xarray is very powerful to extract slice of time ds.sel(time =slice('2000-06-01', '2000-06-10'))

http://xarray.pydata.org/en/stable/time-series.html#datetime-indexing

I hope it could be useful.

Regards Javier Ruano

El mié., 29 abr. 2020 17:34, Javier Ruano javier.ruanno@gmail.com escribió:

pandas doesn't have that problem import pandas as pd x1=pd.DataFrame([['1','2','3']]) x2=pd.DataFrame([['4','5','6']]) pd.concat([x1,x2],axis=1)

El mié., 29 abr. 2020 a las 17:23, Javier Ruano (javier.ruanno@gmail.com) escribió:

the time is the same. do you have tried to change the second one time index?

El mié., 29 abr. 2020 a las 16:36, Xin Zhang (notifications@github.com) escribió:

It's impossible to concatenate two arrays on same named dimensions with different sizes. MCVE Code Sample

import xarray as xrimport pandas as pd

a = xr.DataArray([0], dims=['x']) b = xr.DataArray([1, 2, 3], dims=['x']) a = a.expand_dims("time") b = b.expand_dims("time") a.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) b.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) c = xr.concat([a, b], dim='time')print(c)

Expected Output

[0, 1, 2, 3]

Problem Description

File "C:\Users\Xin\Desktop\test_github.py", line 10, in <module> c = xr.concat([a, b], dim='time') File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 135, in concat return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 455, in _dataarray_concat join=join, File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 319, in _dataset_concat *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py", line 327, in align % (dim, sizes) ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3}

Versions Output of xr.show_versions() INSTALLED VERSIONS

commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3

xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.1.1.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.14.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.4

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO35OSYH6NU6GMCMGWTRPA3PDANCNFSM4MTY32IQ .

pandas doesn't have that problem import pandas as pd x1=pd.DataFrame([['1','2','3']]) x2=pd.DataFrame([['4','5','6']]) pd.concat([x1,x2],axis=1)

El mié., 29 abr. 2020 a las 17:23, Javier Ruano (javier.ruanno@gmail.com) escribió:

the time is the same. do you have tried to change the second one time index?

El mié., 29 abr. 2020 a las 16:36, Xin Zhang (notifications@github.com) escribió:

It's impossible to concatenate two arrays on same named dimensions with different sizes. MCVE Code Sample

import xarray as xrimport pandas as pd

a = xr.DataArray([0], dims=['x']) b = xr.DataArray([1, 2, 3], dims=['x']) a = a.expand_dims("time") b = b.expand_dims("time") a.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) b.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) c = xr.concat([a, b], dim='time')print(c)

Expected Output

[0, 1, 2, 3]

Problem Description

File "C:\Users\Xin\Desktop\test_github.py", line 10, in <module> c = xr.concat([a, b], dim='time') File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 135, in concat return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 455, in _dataarray_concat join=join, File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 319, in _dataset_concat *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py", line 327, in align % (dim, sizes) ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3}

Versions Output of xr.show_versions() INSTALLED VERSIONS

commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3

xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.1.1.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.14.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.4

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO35OSYH6NU6GMCMGWTRPA3PDANCNFSM4MTY32IQ .

the time is the same. do you have tried to change the second one time index?

El mié., 29 abr. 2020 a las 16:36, Xin Zhang (notifications@github.com) escribió:

It's impossible to concatenate two arrays on same named dimensions with different sizes. MCVE Code Sample

import xarray as xrimport pandas as pd

a = xr.DataArray([0], dims=['x']) b = xr.DataArray([1, 2, 3], dims=['x']) a = a.expand_dims("time") b = b.expand_dims("time") a.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) b.coords["time"] = pd.DatetimeIndex(['2020-02-14 05:25:10']) c = xr.concat([a, b], dim='time')print(c)

Expected Output

[0, 1, 2, 3]

Problem Description

File "C:\Users\Xin\Desktop\test_github.py", line 10, in <module> c = xr.concat([a, b], dim='time') File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 135, in concat return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 455, in _dataarray_concat join=join, File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 319, in _dataset_concat *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py", line 327, in align % (dim, sizes) ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3}

Versions Output of xr.show_versions() INSTALLED VERSIONS

commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3

xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.1.1.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.14.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.4

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/4016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO35OSYH6NU6GMCMGWTRPA3PDANCNFSM4MTY32IQ .

I dont try it, but i know your problem. If you try to create from dataarray df.to_dataset(name='participant_A') df.to_dataset(name='participant_B') and after merge them?

xr.merge([ds1, ds2], compat='no_conflicts')

http://xarray.pydata.org/en/stable/combining.html

In potter case you could create nan values to create the same dimensions.

But i have never tried. I found another solution for my data, but it was my alternative.

El dom., 19 abr. 2020 20:57, (Ray) Jinbiao Yang notifications@github.com escribió:

I always use Pandas to deal with my neuroscience data (multi-dimension). It is annoying to stack and unstack all the time and I heard Xarray is designed for multi-dimension data.

In neuroscience research, we usually have multiple participants and we will test them different times, which means the data may look like this:

• participant A:
  • 25100 matrix
• participant B:
  • 25101 matrix

(100 and 101 are the testing times)

But Dataset doesn't support to have 25100 DataArray and 25101 DataArray together. Is there any solution to deal with that kind of data in Xarray?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3984, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO4X4KQA5WPOVUEQQVLRNNCRJANCNFSM4ML467MA .

Yes, but with a lot of information, dask is the only option, and working well with the index. https://github.com/dask/dask/issues/958

El jue., 9 abr. 2020 a las 2:54, Xin Zhang (notifications@github.com) escribió:

@JavierRuano https://github.com/JavierRuano Nice suggestion! I combine them to dataset, convert it to dataframe and then sort_values. Finally, convert the dataframe back to dataset:

ds = cld.to_dataset(name='cld') ds['pair'] = pair

df = ds.to_dataframe() new_ds = df.sort_values(by='cld').to_xarray().transpose()

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3957#issuecomment-611299453, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO73HGCN7EOCCBW3NJDRLU2HHANCNFSM4MD6V32A .

You could access directly to data as ndarray and you could transform dataarray into a dataframe of pandas. Pandas has sort_values. You searched sorting values according z, it is shown in z index.

With more dataArray you could read about Dataset concept...

but i dont develop xarray, i am only user of that module, perhaps you search another type of answer.

http://xarray.pydata.org/en/stable/generated/xarray.Dataset.sortby.html according to values of 1-D dataarrays that share dimension with calling object.

El jue., 9 abr. 2020 4:22, Xin Zhang notifications@github.com escribió:

@JavierRuano https://github.com/JavierRuano Thank you very much. This example is a special case. If the order of z is different for each x and y, do we need to create a tmp DataArray to save the result of looping x and y ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3957#issuecomment-611291129, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO4TP6RSAK7CV3DJV73RLUWPRANCNFSM4MD6V32A .

cld.reindex(z=cld[:,0,0].sortby(cld[:,0,0]).z)

with this solution [0] [1]

<xarray.DataArray (z: 5, y: 2, x: 4)> array([[[ 0. , 1. , 2. , 3. ], [ 4. , 5. , 6. , 7. ]],

   [[ 1. ,  2.5,  4. ,  5.5],
    [ 7. ,  8.5, 10. , 11.5]],

   [[ 8. ,  9. , 10. , 11. ],
    [12. , 13. , 14. , 15. ]],

   [[16. , 17. , 18. , 19. ],
    [20. , 21. , 22. , 23. ]],

   [[24. , 25. , 26. , 27. ],
    [28. , 29. , 30. , 31. ]]])

Coordinates: * z (z) int64 0 4 1 2 3 Dimensions without coordinates: y, x

[0] https://stackoverflow.com/questions/41077393/how-to-sort-the-index-of-a-xarray-dataset-dataarray

[1] https://github.com/pydata/xarray/issues/967

El mié., 8 abr. 2020 a las 14:06, Xin Zhang (notifications@github.com) escribió:

.sortby() only supports sorting DataArray by coords values. I'm trying to sort one DataArray (cld) by data values along one dim and sort another DataArray (pair) by the same order. MCVE Code Sample

import xarray as xrimport numpy as np

x = 4 y = 2 z = 4 data = np.arange(xyz).reshape(z, y, x)

3d array with coords

cld_1 = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)})

2d array without coords

cld_2 = xr.DataArray(np.arange(xy).reshape(y, x)1.5+1, dims=['y', 'x'])

expand 2d to 3d

cld_2 = cld_2.expand_dims(z=[4])

concat

cld = xr.concat([cld_1, cld_2], dim='z')

paired array

pair = cld.copy(data=np.arange(xy(z+1)).reshape(z+1, y, x)) print(cld)print(pair)

Output

<xarray.DataArray (z: 5, y: 2, x: 4)> array([[[ 0. , 1. , 2. , 3. ], [ 4. , 5. , 6. , 7. ]],

   [[ 8. ,  9. , 10. , 11. ],
    [12. , 13. , 14. , 15. ]],

   [[16. , 17. , 18. , 19. ],
    [20. , 21. , 22. , 23. ]],

   [[24. , 25. , 26. , 27. ],
    [28. , 29. , 30. , 31. ]],

   [[ 1. ,  2.5,  4. ,  5.5],
    [ 7. ,  8.5, 10. , 11.5]]])

Coordinates: * z (z) int64 0 1 2 3 4 Dimensions without coordinates: y, x

<xarray.DataArray (z: 5, y: 2, x: 4)> array([[[ 0, 1, 2, 3], [ 4, 5, 6, 7]],

   [[ 8,  9, 10, 11],
    [12, 13, 14, 15]],

   [[16, 17, 18, 19],
    [20, 21, 22, 23]],

   [[24, 25, 26, 27],
    [28, 29, 30, 31]],

   [[32, 33, 34, 35],
    [36, 37, 38, 39]]])

Coordinates: * z (z) int64 0 1 2 3 4 Dimensions without coordinates: y, x

Problem Description

I've tried argsort(): cld.argsort(axis=0), but the result is wrong:

<xarray.DataArray (z: 5, y: 2, x: 4)> array([[[0, 0, 0, 0], [0, 0, 0, 0]],

   [[4, 4, 4, 4],
    [4, 4, 4, 4]],

   [[1, 1, 1, 1],
    [1, 1, 1, 1]],

   [[2, 2, 2, 2],
    [2, 2, 2, 2]],

   [[3, 3, 3, 3],
    [3, 3, 3, 3]]], dtype=int64)

Coordinates: * z (z) int64 0 1 2 3 4 Dimensions without coordinates: y, x

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFOYZYE2UCDJR4AAHAJLRLSAEZANCNFSM4MD6V32A .

import xarray as xr import numpy as np

x = 2 y = 4 z = 3 data = np.arange(xyz).reshape(z, x, y)

3d array with coords

a = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)})

2d array without coords

b = xr.DataArray(np.arange(xy).reshape(x, y)1.5, dims=['y', 'x'])

expand 2d to 3d

b = b.assign_coords({'z':3})

comb = xr.concat([a, b], dim='z')

perhaps you need another thing. http://xarray.pydata.org/en/stable/generated/xarray.concat.html ** consist of variables and coordinates with matching shapes

if you compare your shape are differents a.shape and b.shape Regards. Javier Ruano.

El mié., 8 abr. 2020 a las 1:36, Xin Zhang (notifications@github.com) escribió:

The 3D array has three dims: z, y and x. The 2D array has two dims: y and x. When I try to concatenate them by expanding the 2D array with z dim, there's something wrong in _dataset_concat MCVE Code Sample

import xarray as xrimport numpy as np

x = 2 y = 4 z = 3 data = np.arange(xyz).reshape(z, x, y)

3d array with coords

a = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)})

2d array without coords

b = xr.DataArray(np.arange(xy).reshape(x, y)1.5, dims=['y', 'x'])

expand 2d to 3d

b = b.expand_dims('z')

concat

comb = xr.concat([a, b], dim='z')

Expected Output

Same as np.concatenate:

concat_array = np.concatenate((a, b)) comb = xr.DataArray(concat_array, dims={'z', 'y', 'x'})

<xarray.DataArray (z: 4, x: 2, y: 4)> array([[[ 0. , 1. , 2. , 3. ], [ 4. , 5. , 6. , 7. ]],

   [[ 8. ,  9. , 10. , 11. ],
    [12. , 13. , 14. , 15. ]],

   [[16. , 17. , 18. , 19. ],
    [20. , 21. , 22. , 23. ]],

   [[ 0. ,  1.5,  3. ,  4.5],
    [ 6. ,  7.5,  9. , 10.5]]])

Dimensions without coordinates: z, x, y

Problem Description

comb = xr.concat([a, b], dim='z')

File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 135, in concat return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 455, in _dataarray_concat join=join, File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 395, in _dataset_concat raise ValueError("%r is not present in all datasets." % k) ValueError: 'z' is not present in all datasets.

If I change b = b.expand_dims('z') to b = b.expand_dims(z=3) as suggested by @dcherian https://github.com/dcherian , I still get the same error. Versions Output of xr.show_versions() INSTALLED VERSIONS

commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: None libnetcdf: None

xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.1.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.14.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.4

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIGDFO45LDV75NEZWY5FKLDRLPILFANCNFSM4MDRZJZQ .

df1=xarray.open_mfdataset plus parallel=True df1=df1.rename({'xarray_dataarray_variable':'v'})

The chunksize changes to 365 and the dataset creates inside a Datarray of 365, not the global size which is 14610. [xarray.Dataset('u':df1,'v'df2)]

df2 same operation

Pseudo Solution for me. xarray.Dataset({'u':df1.u.chunk(14610),'v':df2.v.chunk(14610)},coords={'time':time_Index,'latitude':latitude_Index,'longitude':longitude_Index,'level':level_Index})

backends\api.py DATAARRAY_NAME = "xarray_dataarray_name" DATAARRAY_VARIABLE = "xarray_dataarray_variable"

The name is auto when i open the file ,xarrray.open_dataset( parallel=True)

core\dataarray.py

def rename( self, new_name_or_name_dict: Union[Hashable, Mapping[Hashable, Hashable]] = None, **names: Hashable, ) -> "DataArray":

    if names or utils.is_dict_like(new_name_or_name_dict):
        new_name_or_name_dict = cast(
            Mapping[Hashable, Hashable], new_name_or_name_dict
        )
        name_dict = either_dict_or_kwargs(new_name_or_name_dict, names, "rename")
        **dataset = self.**_to_temp_dataset()**.rename(name_dict)
        return self._from_temp_dataset(dataset)**
    else:
        new_name_or_name_dict = cast(Hashable, new_name_or_name_dict)
        return self._replace(name=new_name_or_name_dict)

I think an operation changing the previous chunksize or something with xarrray.open_dataset parallel=True, (core\parallel.py) because the chunksize changes to 365 based on days of the year Sorry i cannot help more.

After i modify again the chunksize to 365 to avoid the Memory Error <xarray.Dataset> Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610) Data variables: u (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(365, 47, 68, 81)> v (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(365, 47, 68, 81)>

The strange is the DataArray chunksize changes after i modify the name 'xarray_dataarray_variable' to use another to the Dataset.

<xarray.Dataset> Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610) Data variables: u (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(14610, 47, 68, 81)> v (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(14610, 47, 68, 81)>