html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/5208#issuecomment-829638401,https://api.github.com/repos/pydata/xarray/issues/5208,829638401,MDEyOklzc3VlQ29tbWVudDgyOTYzODQwMQ==,34353851,2021-04-29T22:33:05Z,2021-04-29T22:33:05Z,NONE,"import numpy as np import xarray as xr # Creates DataArrays nt = 4 time = np.arange (nt) * 86400.0 time = xr.DataArray (time, coords=[time,], dims=[""time"",]) aa = time * 2.0 # Adding attributes to DataArrays time.attrs['units'] = ""second"" aa.attrs['units'] = ""whatever"" # Attributes are visible in the DataArrays print ('----------> time DataArray: ') print (time) print ('----------> aa DataArray : ' ) print (aa) print ('----------> aa attributes : ') print (aa.attrs ) # Creating a Dataset ds = xr.Dataset( { ""aa"": ([""time"",], aa), }, coords={""time"": ([""time"",], time), }, ) # Attributes are not visible in the Dataset print ('----------> DataSet before setting attributes') print (ds) # My request #1 : attributes of the DataArrays should be added to the DataSet (may be optional) print ('----------> Attributes of aa in DataSet : none') print ( ds['aa'].attrs ) print ('----------> Attributes of aa outside DataSet : still here') print ( aa.attrs ) print ('----------> Attributes are not written to the NetCDF file') ds.to_netcdf ('sample1.nc') # Adding attributes directly to the Dataset # Attributes are still not visible in the Dataset print ('----------> DataSet after setting attributes : attributes not shown' ) ds=ds.assign_attrs({'Visible':'NotInvisibleMan'}) ds['time'].attrs['units']=""second"" ds['aa'].attrs['units']=""whatever"" ds.to_netcdf('safeReturn.nc') print(xr.open_dataset('safeReturn.nc').attrs) print(xr.open_dataset('safeReturn.nc')['aa'].attrs) print(xr.open_dataset('safeReturn.nc')['time'].attrs)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,865003095 https://github.com/pydata/xarray/issues/5225#issuecomment-828306616,https://api.github.com/repos/pydata/xarray/issues/5225,828306616,MDEyOklzc3VlQ29tbWVudDgyODMwNjYxNg==,34353851,2021-04-28T09:31:22Z,2021-04-28T09:31:22Z,NONE,I'm sorry I couldn't be more helpful this time,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,869180122 https://github.com/pydata/xarray/issues/5225#issuecomment-828256725,https://api.github.com/repos/pydata/xarray/issues/5225,828256725,MDEyOklzc3VlQ29tbWVudDgyODI1NjcyNQ==,34353851,2021-04-28T08:23:43Z,2021-04-28T08:23:43Z,NONE,"It works again, i have tried to save a netcdf file what produces the bug, but nothing. The traceback was from debug mode of django. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,869180122 https://github.com/pydata/xarray/issues/5225#issuecomment-828002269,https://api.github.com/repos/pydata/xarray/issues/5225,828002269,MDEyOklzc3VlQ29tbWVudDgyODAwMjI2OQ==,34353851,2021-04-27T23:00:32Z,2021-04-27T23:00:32Z,NONE,"Linux streamDebian 5.10.0-6-amd64 #1 SMP Debian 5.10.28-1 (2021-04-09) x86_64 GNU/Linux >>> xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] python-bits: 64 OS: Linux OS-release: 5.10.0-6-amd64 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.17.0 pandas: 1.2.4 numpy: 1.19.5 scipy: 1.6.3 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.04.1 distributed: None matplotlib: 3.4.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 52.0.0 pip: 20.3.4 conda: None pytest: 6.0.2 IPython: 7.20.0 sphinx: None ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,869180122 https://github.com/pydata/xarray/issues/5085#issuecomment-822295458,https://api.github.com/repos/pydata/xarray/issues/5085,822295458,MDEyOklzc3VlQ29tbWVudDgyMjI5NTQ1OA==,34353851,2021-04-19T08:52:19Z,2021-04-19T08:52:19Z,NONE,"Thanks for your quick response, it is true that you have greatly improved the examples in the documentation, I do not know to what extent it would solve any lack. The np.ufunc examples seemed insufficient a year ago but you have already solved. And it is always better to be aware of the improvements that you are introducing before writing new documentation. Thanks for your attention.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,842610988 https://github.com/pydata/xarray/issues/5085#issuecomment-822066185,https://api.github.com/repos/pydata/xarray/issues/5085,822066185,MDEyOklzc3VlQ29tbWVudDgyMjA2NjE4NQ==,34353851,2021-04-18T21:39:38Z,2021-04-18T21:39:38Z,NONE,"Shall i close this issue? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,842610988 https://github.com/pydata/xarray/issues/5085#issuecomment-815276368,https://api.github.com/repos/pydata/xarray/issues/5085,815276368,MDEyOklzc3VlQ29tbWVudDgxNTI3NjM2OA==,34353851,2021-04-07T21:27:28Z,2021-04-07T21:46:39Z,NONE,"Ok, thanks @max-sixty and @keewis . I hope it is useful for the proper and efficient use of xarray. you know better what development path it is taking.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,842610988 https://github.com/pydata/xarray/issues/5085#issuecomment-814932746,https://api.github.com/repos/pydata/xarray/issues/5085,814932746,MDEyOklzc3VlQ29tbWVudDgxNDkzMjc0Ng==,34353851,2021-04-07T13:52:38Z,2021-04-07T13:52:38Z,NONE,"Hi again @max-sixty , thanks for your advice, sure, the outputs have a lot of sense. I think this new example has the good structure like another examples. https://github.com/JavierRuano/ASI_Steady/blob/main/Examples/AirStagnationIndex_Wang_Xarray_Example.ipynb Regards Javier Ruano.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,842610988 https://github.com/pydata/xarray/issues/5085#issuecomment-814462311,https://api.github.com/repos/pydata/xarray/issues/5085,814462311,MDEyOklzc3VlQ29tbWVudDgxNDQ2MjMxMQ==,34353851,2021-04-06T21:53:31Z,2021-04-06T21:53:31Z,NONE,"I think it is interesting the operations over the time axis with numpy reduce, and the ufuncs operations. From my point of view there are a lot of pandas and dask users who could learn to use xarray with that example or what they are using netcdf-4 and numpy directly. I have created the example from the library, and it is not problem to change it. But it shows a stagnation calculation, it was the intention. But i understand what xarray is climate focused and you know if the topic is interesting and the example is really useful for your project. For me xarray have been very useful. If you prefer another type of example, we could refactor it. Regards Javier Ruano.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,842610988 https://github.com/pydata/xarray/issues/4085#issuecomment-631636660,https://api.github.com/repos/pydata/xarray/issues/4085,631636660,MDEyOklzc3VlQ29tbWVudDYzMTYzNjY2MA==,34353851,2020-05-20T18:07:22Z,2020-05-20T18:07:22Z,NONE,"I use http://xarray.pydata.org/en/stable/generated/xarray.apply_ufunc.html because it is faster. El mié., 20 may. 2020 a las 20:01, Javier Ruano () escribió: > if you append compute() it should not be a lazy operation. > But my advice is like user only. > > El mié., 20 may. 2020 a las 19:51, Rob Hetland () > escribió: > >> I have a large DataSet, including these DataArrays: >> >> >> dask.array >> >> and >> >> >> dask.array >> >> (The coordinates and attributes excluded for brevity, but they match in >> the right ways.) >> >> When I do math operations with the 4D DataArray (temp) and 3D DataArray >> (zeta), no problem: >> >> ds.zeta * ds.temp >> >> >> dask.array >> >> This returns an object instantly, and the result is lazily evaluated. >> However, if I just try to add temp to itself, >> >> ds.temp + ds.temp >> >> this fails (eventually) as my medium sized computer runs out of memory, >> since it starts to evaluate the numbers as if I did a compute() or asked >> for the values. Note 2*ds.temp or ds.temp**2 is lazily evaluated, and >> returns an object instantly. Chunk size does not seem to be an issue, as I >> have tried a number of reasonable choices without success. >> >> Why can't such simple math operations between two large arrays also be >> lazily evaluated? >> >> — >> You are receiving this because you are subscribed to this thread. >> Reply to this email directly, view it on GitHub >> , or unsubscribe >> >> . >> > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474 https://github.com/pydata/xarray/issues/4085#issuecomment-631633988,https://api.github.com/repos/pydata/xarray/issues/4085,631633988,MDEyOklzc3VlQ29tbWVudDYzMTYzMzk4OA==,34353851,2020-05-20T18:01:42Z,2020-05-20T18:01:42Z,NONE,"if you append compute() it should not be a lazy operation. But my advice is like user only. El mié., 20 may. 2020 a las 19:51, Rob Hetland () escribió: > I have a large DataSet, including these DataArrays: > > > dask.array > > and > > > dask.array > > (The coordinates and attributes excluded for brevity, but they match in > the right ways.) > > When I do math operations with the 4D DataArray (temp) and 3D DataArray > (zeta), no problem: > > ds.zeta * ds.temp > > > dask.array > > This returns an object instantly, and the result is lazily evaluated. > However, if I just try to add temp to itself, > > ds.temp + ds.temp > > this fails (eventually) as my medium sized computer runs out of memory, > since it starts to evaluate the numbers as if I did a compute() or asked > for the values. Note 2*ds.temp or ds.temp**2 is lazily evaluated, and > returns an object instantly. Chunk size does not seem to be an issue, as I > have tried a number of reasonable choices without success. > > Why can't such simple math operations between two large arrays also be > lazily evaluated? > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474 https://github.com/pydata/xarray/issues/4016#issuecomment-624384220,https://api.github.com/repos/pydata/xarray/issues/4016,624384220,MDEyOklzc3VlQ29tbWVudDYyNDM4NDIyMA==,34353851,2020-05-06T00:54:38Z,2020-05-06T00:54:38Z,NONE,"I think another solution you could add an aleat microsecond or nanosecond field ... in the datetime index and the index should be different. You could test if there are colision in the index (it means the index<->time is not empty like the sort algorithm, something similar to https://en.wikipedia.org/wiki/Radix_sort) It is like the reverse of this solution https://stackoverflow.com/questions/32827169/python-reduce-precision-pandas-timestamp-dataframe/32827432 Add another coordinate could be overloaded and xarray is very powerful to extract slice of time ds.sel(time =slice('2000-06-01', '2000-06-10')) http://xarray.pydata.org/en/stable/time-series.html#datetime-indexing I hope it could be useful. Regards Javier Ruano El mié., 29 abr. 2020 17:34, Javier Ruano escribió: > pandas doesn't have that problem > import pandas as pd > x1=pd.DataFrame([['1','2','3']]) > x2=pd.DataFrame([['4','5','6']]) > pd.concat([x1,x2],axis=1) > > El mié., 29 abr. 2020 a las 17:23, Javier Ruano () > escribió: > >> the time is the same. do you have tried to change the second one time >> index? >> >> El mié., 29 abr. 2020 a las 16:36, Xin Zhang () >> escribió: >> >>> It's impossible to concatenate two arrays on same named dimensions with >>> different sizes. >>> MCVE Code Sample >>> >>> import xarray as xrimport pandas as pd >>> >>> a = xr.DataArray([0], dims=['x']) >>> b = xr.DataArray([1, 2, 3], dims=['x']) >>> a = a.expand_dims(""time"") >>> b = b.expand_dims(""time"") >>> a.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) >>> b.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) >>> c = xr.concat([a, b], dim='time')print(c) >>> >>> Expected Output >>> >>> [0, 1, 2, 3] >>> >>> Problem Description >>> >>> File ""C:\Users\Xin\Desktop\test_github.py"", line 10, in >>> c = xr.concat([a, b], dim='time') >>> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 135, in concat >>> return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) >>> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 455, in _dataarray_concat >>> join=join, >>> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 319, in _dataset_concat >>> *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value >>> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py"", line 327, in align >>> % (dim, sizes) >>> ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3} >>> >>> Versions Output of xr.show_versions() INSTALLED VERSIONS >>> >>> commit: None >>> python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, >>> 21:48:41) [MSC v.1916 64 bit (AMD64)] >>> python-bits: 64 >>> OS: Windows >>> OS-release: 10 >>> machine: AMD64 >>> processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel >>> byteorder: little >>> LC_ALL: None >>> LANG: None >>> LOCALE: None.None >>> libhdf5: 1.10.5 >>> libnetcdf: 4.7.3 >>> >>> xarray: 0.15.1 >>> pandas: 1.0.3 >>> numpy: 1.18.1 >>> scipy: 1.4.1 >>> netCDF4: 1.5.3 >>> pydap: None >>> h5netcdf: None >>> h5py: 2.10.0 >>> Nio: None >>> zarr: 2.4.0 >>> cftime: 1.1.1.2 >>> nc_time_axis: None >>> PseudoNetCDF: None >>> rasterio: 1.1.3 >>> cfgrib: None >>> iris: None >>> bottleneck: None >>> dask: 2.10.1 >>> distributed: 2.14.0 >>> matplotlib: 3.2.1 >>> cartopy: 0.17.0 >>> seaborn: 0.10.0 >>> numbagg: None >>> setuptools: 46.1.3.post20200325 >>> pip: 20.0.2 >>> conda: None >>> pytest: None >>> IPython: 7.13.0 >>> sphinx: 2.4.4 >>> >>> — >>> You are receiving this because you are subscribed to this thread. >>> Reply to this email directly, view it on GitHub >>> , or unsubscribe >>> >>> . >>> >> ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,609108666 https://github.com/pydata/xarray/issues/4016#issuecomment-621290319,https://api.github.com/repos/pydata/xarray/issues/4016,621290319,MDEyOklzc3VlQ29tbWVudDYyMTI5MDMxOQ==,34353851,2020-04-29T15:35:01Z,2020-04-29T15:35:01Z,NONE,"pandas doesn't have that problem import pandas as pd x1=pd.DataFrame([['1','2','3']]) x2=pd.DataFrame([['4','5','6']]) pd.concat([x1,x2],axis=1) El mié., 29 abr. 2020 a las 17:23, Javier Ruano () escribió: > the time is the same. do you have tried to change the second one time > index? > > El mié., 29 abr. 2020 a las 16:36, Xin Zhang () > escribió: > >> It's impossible to concatenate two arrays on same named dimensions with >> different sizes. >> MCVE Code Sample >> >> import xarray as xrimport pandas as pd >> >> a = xr.DataArray([0], dims=['x']) >> b = xr.DataArray([1, 2, 3], dims=['x']) >> a = a.expand_dims(""time"") >> b = b.expand_dims(""time"") >> a.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) >> b.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) >> c = xr.concat([a, b], dim='time')print(c) >> >> Expected Output >> >> [0, 1, 2, 3] >> >> Problem Description >> >> File ""C:\Users\Xin\Desktop\test_github.py"", line 10, in >> c = xr.concat([a, b], dim='time') >> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 135, in concat >> return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) >> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 455, in _dataarray_concat >> join=join, >> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 319, in _dataset_concat >> *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value >> File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py"", line 327, in align >> % (dim, sizes) >> ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3} >> >> Versions Output of xr.show_versions() INSTALLED VERSIONS >> >> commit: None >> python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) >> [MSC v.1916 64 bit (AMD64)] >> python-bits: 64 >> OS: Windows >> OS-release: 10 >> machine: AMD64 >> processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel >> byteorder: little >> LC_ALL: None >> LANG: None >> LOCALE: None.None >> libhdf5: 1.10.5 >> libnetcdf: 4.7.3 >> >> xarray: 0.15.1 >> pandas: 1.0.3 >> numpy: 1.18.1 >> scipy: 1.4.1 >> netCDF4: 1.5.3 >> pydap: None >> h5netcdf: None >> h5py: 2.10.0 >> Nio: None >> zarr: 2.4.0 >> cftime: 1.1.1.2 >> nc_time_axis: None >> PseudoNetCDF: None >> rasterio: 1.1.3 >> cfgrib: None >> iris: None >> bottleneck: None >> dask: 2.10.1 >> distributed: 2.14.0 >> matplotlib: 3.2.1 >> cartopy: 0.17.0 >> seaborn: 0.10.0 >> numbagg: None >> setuptools: 46.1.3.post20200325 >> pip: 20.0.2 >> conda: None >> pytest: None >> IPython: 7.13.0 >> sphinx: 2.4.4 >> >> — >> You are receiving this because you are subscribed to this thread. >> Reply to this email directly, view it on GitHub >> , or unsubscribe >> >> . >> > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,609108666 https://github.com/pydata/xarray/issues/4016#issuecomment-621283446,https://api.github.com/repos/pydata/xarray/issues/4016,621283446,MDEyOklzc3VlQ29tbWVudDYyMTI4MzQ0Ng==,34353851,2020-04-29T15:23:59Z,2020-04-29T15:23:59Z,NONE,"the time is the same. do you have tried to change the second one time index? El mié., 29 abr. 2020 a las 16:36, Xin Zhang () escribió: > It's impossible to concatenate two arrays on same named dimensions with > different sizes. > MCVE Code Sample > > import xarray as xrimport pandas as pd > > a = xr.DataArray([0], dims=['x']) > b = xr.DataArray([1, 2, 3], dims=['x']) > a = a.expand_dims(""time"") > b = b.expand_dims(""time"") > a.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) > b.coords[""time""] = pd.DatetimeIndex(['2020-02-14 05:25:10']) > c = xr.concat([a, b], dim='time')print(c) > > Expected Output > > [0, 1, 2, 3] > > Problem Description > > File ""C:\Users\Xin\Desktop\test_github.py"", line 10, in > c = xr.concat([a, b], dim='time') > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 135, in concat > return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 455, in _dataarray_concat > join=join, > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 319, in _dataset_concat > *datasets, join=join, copy=False, exclude=[dim], fill_value=fill_value > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\alignment.py"", line 327, in align > % (dim, sizes) > ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {1, 3} > > Versions Output of xr.show_versions() INSTALLED VERSIONS > > commit: None > python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) > [MSC v.1916 64 bit (AMD64)] > python-bits: 64 > OS: Windows > OS-release: 10 > machine: AMD64 > processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel > byteorder: little > LC_ALL: None > LANG: None > LOCALE: None.None > libhdf5: 1.10.5 > libnetcdf: 4.7.3 > > xarray: 0.15.1 > pandas: 1.0.3 > numpy: 1.18.1 > scipy: 1.4.1 > netCDF4: 1.5.3 > pydap: None > h5netcdf: None > h5py: 2.10.0 > Nio: None > zarr: 2.4.0 > cftime: 1.1.1.2 > nc_time_axis: None > PseudoNetCDF: None > rasterio: 1.1.3 > cfgrib: None > iris: None > bottleneck: None > dask: 2.10.1 > distributed: 2.14.0 > matplotlib: 3.2.1 > cartopy: 0.17.0 > seaborn: 0.10.0 > numbagg: None > setuptools: 46.1.3.post20200325 > pip: 20.0.2 > conda: None > pytest: None > IPython: 7.13.0 > sphinx: 2.4.4 > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,609108666 https://github.com/pydata/xarray/issues/3984#issuecomment-616214519,https://api.github.com/repos/pydata/xarray/issues/3984,616214519,MDEyOklzc3VlQ29tbWVudDYxNjIxNDUxOQ==,34353851,2020-04-19T19:49:19Z,2020-04-19T19:49:19Z,NONE,"I dont try it, but i know your problem. If you try to create from dataarray df.to_dataset(name='participant_A') df.to_dataset(name='participant_B') and after merge them? xr.merge([ds1, ds2], compat='no_conflicts') http://xarray.pydata.org/en/stable/combining.html In potter case you could create nan values to create the same dimensions. But i have never tried. I found another solution for my data, but it was my alternative. El dom., 19 abr. 2020 20:57, (Ray) Jinbiao Yang escribió: > I always use Pandas to deal with my neuroscience data (multi-dimension). > It is annoying to stack and unstack all the time and I heard Xarray is > designed for multi-dimension data. > > In neuroscience research, we usually have multiple participants and we > will test them different times, which means the data may look like this: > > - participant A: > - 2*5*100 matrix > - participant B: > - 2*5*101 matrix > > (100 and 101 are the testing times) > > But *Dataset doesn't support to have 2*5*100 DataArray and 2*5*101 > DataArray together*. Is there any solution to deal with that kind of data > in Xarray? > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602793814 https://github.com/pydata/xarray/issues/3957#issuecomment-611302006,https://api.github.com/repos/pydata/xarray/issues/3957,611302006,MDEyOklzc3VlQ29tbWVudDYxMTMwMjAwNg==,34353851,2020-04-09T03:04:53Z,2020-04-09T03:04:53Z,NONE,"Yes, but with a lot of information, dask is the only option, and working well with the index. https://github.com/dask/dask/issues/958 El jue., 9 abr. 2020 a las 2:54, Xin Zhang () escribió: > @JavierRuano Nice suggestion! I combine > them to dataset, convert it to dataframe and then sort_values. Finally, > convert the dataframe back to dataset: > > ds = cld.to_dataset(name='cld') > ds['pair'] = pair > > df = ds.to_dataframe() > new_ds = df.sort_values(by='cld').to_xarray().transpose() > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,596606599 https://github.com/pydata/xarray/issues/3957#issuecomment-611295039,https://api.github.com/repos/pydata/xarray/issues/3957,611295039,MDEyOklzc3VlQ29tbWVudDYxMTI5NTAzOQ==,34353851,2020-04-09T02:36:56Z,2020-04-09T02:36:56Z,NONE,"You could access directly to data as ndarray and you could transform dataarray into a dataframe of pandas. Pandas has sort_values. You searched sorting values according z, it is shown in z index. With more dataArray you could read about Dataset concept... but i dont develop xarray, i am only user of that module, perhaps you search another type of answer. http://xarray.pydata.org/en/stable/generated/xarray.Dataset.sortby.html according to values of 1-D dataarrays that share dimension with calling object. El jue., 9 abr. 2020 4:22, Xin Zhang escribió: > @JavierRuano Thank you very much. This > example is a special case. If the order of z is different for each x and y, > do we need to create a tmp DataArray to save the result of looping x and y > ? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,596606599 https://github.com/pydata/xarray/issues/3957#issuecomment-611047964,https://api.github.com/repos/pydata/xarray/issues/3957,611047964,MDEyOklzc3VlQ29tbWVudDYxMTA0Nzk2NA==,34353851,2020-04-08T16:08:00Z,2020-04-08T16:08:00Z,NONE,"cld.reindex(z=cld[:,0,0].sortby(cld[:,0,0]).z) with this solution [0] [1] array([[[ 0. , 1. , 2. , 3. ], [ 4. , 5. , 6. , 7. ]], [[ 1. , 2.5, 4. , 5.5], [ 7. , 8.5, 10. , 11.5]], [[ 8. , 9. , 10. , 11. ], [12. , 13. , 14. , 15. ]], [[16. , 17. , 18. , 19. ], [20. , 21. , 22. , 23. ]], [[24. , 25. , 26. , 27. ], [28. , 29. , 30. , 31. ]]]) Coordinates: * z (z) int64 0 4 1 2 3 Dimensions without coordinates: y, x [0] https://stackoverflow.com/questions/41077393/how-to-sort-the-index-of-a-xarray-dataset-dataarray [1] https://github.com/pydata/xarray/issues/967 El mié., 8 abr. 2020 a las 14:06, Xin Zhang () escribió: > .sortby() only supports sorting DataArray by coords values. I'm trying to > sort one DataArray (cld) by data values along one dim and sort another > DataArray (pair) by the same order. > MCVE Code Sample > > import xarray as xrimport numpy as np > > x = 4 > y = 2 > z = 4 > data = np.arange(x*y*z).reshape(z, y, x) > # 3d array with coords > cld_1 = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)}) > # 2d array without coords > cld_2 = xr.DataArray(np.arange(x*y).reshape(y, x)*1.5+1, dims=['y', 'x']) > # expand 2d to 3d > cld_2 = cld_2.expand_dims(z=[4]) > # concat > cld = xr.concat([cld_1, cld_2], dim='z') > # paired array > pair = cld.copy(data=np.arange(x*y*(z+1)).reshape(z+1, y, x)) > print(cld)print(pair) > > Output > > > array([[[ 0. , 1. , 2. , 3. ], > [ 4. , 5. , 6. , 7. ]], > > [[ 8. , 9. , 10. , 11. ], > [12. , 13. , 14. , 15. ]], > > [[16. , 17. , 18. , 19. ], > [20. , 21. , 22. , 23. ]], > > [[24. , 25. , 26. , 27. ], > [28. , 29. , 30. , 31. ]], > > [[ 1. , 2.5, 4. , 5.5], > [ 7. , 8.5, 10. , 11.5]]]) > Coordinates: > * z (z) int64 0 1 2 3 4 > Dimensions without coordinates: y, x > > > array([[[ 0, 1, 2, 3], > [ 4, 5, 6, 7]], > > [[ 8, 9, 10, 11], > [12, 13, 14, 15]], > > [[16, 17, 18, 19], > [20, 21, 22, 23]], > > [[24, 25, 26, 27], > [28, 29, 30, 31]], > > [[32, 33, 34, 35], > [36, 37, 38, 39]]]) > Coordinates: > * z (z) int64 0 1 2 3 4 > Dimensions without coordinates: y, x > > Problem Description > > I've tried argsort(): cld.argsort(axis=0), but the result is wrong: > > > array([[[0, 0, 0, 0], > [0, 0, 0, 0]], > > [[4, 4, 4, 4], > [4, 4, 4, 4]], > > [[1, 1, 1, 1], > [1, 1, 1, 1]], > > [[2, 2, 2, 2], > [2, 2, 2, 2]], > > [[3, 3, 3, 3], > [3, 3, 3, 3]]], dtype=int64) > Coordinates: > * z (z) int64 0 1 2 3 4 > Dimensions without coordinates: y, x > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,596606599 https://github.com/pydata/xarray/issues/3954#issuecomment-610715885,https://api.github.com/repos/pydata/xarray/issues/3954,610715885,MDEyOklzc3VlQ29tbWVudDYxMDcxNTg4NQ==,34353851,2020-04-08T02:25:34Z,2020-04-08T02:25:34Z,NONE,"import xarray as xr import numpy as np x = 2 y = 4 z = 3 data = np.arange(x*y*z).reshape(z, x, y) # 3d array with coords a = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)}) # 2d array without coords b = xr.DataArray(np.arange(x*y).reshape(x, y)*1.5, dims=['y', 'x']) # expand 2d to 3d b = b.assign_coords({'z':3}) comb = xr.concat([a, b], dim='z') perhaps you need another thing. http://xarray.pydata.org/en/stable/generated/xarray.concat.html ** consist of variables and coordinates with matching shapes if you compare your shape are differents a.shape and b.shape Regards. Javier Ruano. El mié., 8 abr. 2020 a las 1:36, Xin Zhang () escribió: > The 3D array has three dims: z, y and x. > The 2D array has two dims: y and x. > When I try to concatenate them by expanding the 2D array with z dim, > there's something wrong in _dataset_concat > MCVE Code Sample > > import xarray as xrimport numpy as np > > x = 2 > y = 4 > z = 3 > data = np.arange(x*y*z).reshape(z, x, y) > # 3d array with coords > a = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)}) > # 2d array without coords > b = xr.DataArray(np.arange(x*y).reshape(x, y)*1.5, dims=['y', 'x']) > # expand 2d to 3d > b = b.expand_dims('z') > # concat > comb = xr.concat([a, b], dim='z') > > Expected Output > > Same as np.concatenate: > > concat_array = np.concatenate((a, b)) > comb = xr.DataArray(concat_array, dims={'z', 'y', 'x'}) > > > array([[[ 0. , 1. , 2. , 3. ], > [ 4. , 5. , 6. , 7. ]], > > [[ 8. , 9. , 10. , 11. ], > [12. , 13. , 14. , 15. ]], > > [[16. , 17. , 18. , 19. ], > [20. , 21. , 22. , 23. ]], > > [[ 0. , 1.5, 3. , 4.5], > [ 6. , 7.5, 9. , 10.5]]]) > Dimensions without coordinates: z, x, y > > Problem Description > > comb = xr.concat([a, b], dim='z') > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 135, in concat > return f(objs, dim, data_vars, coords, compat, positions, fill_value, join) > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 455, in _dataarray_concat > join=join, > File ""E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py"", line 395, in _dataset_concat > raise ValueError(""%r is not present in all datasets."" % k) > ValueError: 'z' is not present in all datasets. > > If I change b = b.expand_dims('z') to b = b.expand_dims(z=3) as suggested > by @dcherian , I still get the same error. > Versions Output of `xr.show_versions()` INSTALLED VERSIONS > > commit: None > python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) > [MSC v.1916 64 bit (AMD64)] > python-bits: 64 > OS: Windows > OS-release: 10 > machine: AMD64 > processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel > byteorder: little > LC_ALL: None > LANG: None > LOCALE: None.None > libhdf5: None > libnetcdf: None > > xarray: 0.15.1 > pandas: 1.0.3 > numpy: 1.18.1 > scipy: 1.4.1 > netCDF4: None > pydap: None > h5netcdf: None > h5py: None > Nio: None > zarr: None > cftime: 1.1.1.2 > nc_time_axis: None > PseudoNetCDF: None > rasterio: 1.1.3 > cfgrib: None > iris: None > bottleneck: None > dask: 2.10.1 > distributed: 2.14.0 > matplotlib: 3.2.1 > cartopy: 0.17.0 > seaborn: 0.10.0 > numbagg: None > setuptools: 46.1.3.post20200325 > pip: 20.0.2 > conda: None > pytest: None > IPython: 7.13.0 > sphinx: 2.4.4 > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,596249070 https://github.com/pydata/xarray/issues/3795#issuecomment-590375940,https://api.github.com/repos/pydata/xarray/issues/3795,590375940,MDEyOklzc3VlQ29tbWVudDU5MDM3NTk0MA==,34353851,2020-02-24T15:21:42Z,2020-02-24T15:26:57Z,NONE,"df1=xarray.open_mfdataset plus parallel=True df1=df1.rename({'__xarray_dataarray_variable__':'v'}) The chunksize changes to 365 and the dataset creates inside a Datarray of 365, not the global size which is 14610. [xarray.Dataset('u':df1,'v'df2)] df2 same operation Pseudo Solution for me. xarray.Dataset({'u':df1.u.chunk(14610),'v':df2.v.chunk(14610)},coords={'time':time_Index,'latitude':latitude_Index,'longitude':longitude_Index,'level':level_Index})","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418 https://github.com/pydata/xarray/issues/3795#issuecomment-590351872,https://api.github.com/repos/pydata/xarray/issues/3795,590351872,MDEyOklzc3VlQ29tbWVudDU5MDM1MTg3Mg==,34353851,2020-02-24T14:39:11Z,2020-02-24T15:06:16Z,NONE," backends\api.py DATAARRAY_NAME = ""__xarray_dataarray_name__"" DATAARRAY_VARIABLE = ""__xarray_dataarray_variable__"" The name is auto when i open the file ,xarrray.open_dataset( parallel=True) core\dataarray.py def rename( self, new_name_or_name_dict: Union[Hashable, Mapping[Hashable, Hashable]] = None, **names: Hashable, ) -> ""DataArray"": if names or utils.is_dict_like(new_name_or_name_dict): new_name_or_name_dict = cast( Mapping[Hashable, Hashable], new_name_or_name_dict ) name_dict = either_dict_or_kwargs(new_name_or_name_dict, names, ""rename"") **dataset = self.**_to_temp_dataset()**.rename(name_dict) return self._from_temp_dataset(dataset)** else: new_name_or_name_dict = cast(Hashable, new_name_or_name_dict) return self._replace(name=new_name_or_name_dict) I think an operation changing the previous chunksize or something with xarrray.open_dataset parallel=True, (core\parallel.py) because the chunksize changes to 365 based on days of the year Sorry i cannot help more. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418 https://github.com/pydata/xarray/issues/3795#issuecomment-590297372,https://api.github.com/repos/pydata/xarray/issues/3795,590297372,MDEyOklzc3VlQ29tbWVudDU5MDI5NzM3Mg==,34353851,2020-02-24T12:30:34Z,2020-02-24T12:30:34Z,NONE,"After i modify again the chunksize to 365 to avoid the Memory Error Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610) Data variables: u (time, level, latitude, longitude) float32 dask.array v (time, level, latitude, longitude) float32 dask.array","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418 https://github.com/pydata/xarray/issues/3795#issuecomment-590294555,https://api.github.com/repos/pydata/xarray/issues/3795,590294555,MDEyOklzc3VlQ29tbWVudDU5MDI5NDU1NQ==,34353851,2020-02-24T12:21:39Z,2020-02-24T12:21:39Z,NONE,"The strange is the DataArray chunksize changes after i modify the name '__xarray_dataarray_variable__' to use another to the Dataset. Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610) Data variables: u (time, level, latitude, longitude) float32 dask.array v (time, level, latitude, longitude) float32 dask.array","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418