id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1412846601,I_kwDOAMm_X85UNlQJ,7180,open_mfdataset for merging of netCDF files with different time values,761677,closed,0,,,2,2022-10-18T09:06:14Z,2022-10-19T08:47:25Z,2022-10-19T08:47:25Z,NONE,,,,"### What is your issue? In xarray version 0.16 I could combinine multiple netCDF files with different values on the time coordinate along a new dimension in a straight forward way: ``` def rename_time(ds: xr.Dataset) -> xr.Dataset: return ds.rename_dims({'time': 'sim_time'}) ds = xr.open_mfdataset(ncfiles, concat_dim='simulation', preprocess=rename_time, combine='nested', coords=['time']) ``` The merged dataset got a new time dimension called ""sim_time"" which had dimensions of simulation and time, so a 2D array.: ![old_and_good](https://user-images.githubusercontent.com/761677/196384272-12418a69-af0e-4660-af2b-24840e0bfbf5.png) When upgrading xarray to 2022.9.0, this code no longer works: ``` ds_allsims = xr.open_mfdataset(coarser_files, concat_dim='simulation', preprocess=rename_time, File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/backends/api.py"", line 1003, in open_mfdataset combined = _nested_combine( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/combine.py"", line 365, in _nested_combine combined = _combine_nd( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/combine.py"", line 239, in _combine_nd combined_ids = _combine_all_along_first_dim( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/combine.py"", line 275, in _combine_all_along_first_dim new_combined_ids[new_id] = _combine_1d( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/combine.py"", line 298, in _combine_1d combined = concat( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/concat.py"", line 243, in concat return _dataset_concat( File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/concat.py"", line 569, in _dataset_concat combined_idx = indexes[0].concat(indexes, dim, positions) File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/indexes.py"", line 327, in concat new_pd_index = cls._concat_indexes(indexes, dim, positions) File ""/home/jorgens/mambaforge/envs/era_acute_post_processor/lib/python3.10/site-packages/xarray/core/indexes.py"", line 307, in _concat_indexes raise ValueError( ValueError: Cannot concatenate along dimension 'simulation' indexes with dimensions: 'sim_time' ``` Looking around, I found this answer on stack overflow for a similar question: https://stackoverflow.com/a/55500434 I had to modify the preprocessor in the answer to make it work: ``` def rename_time(ds: xr.Dataset) -> xr.Dataset: ds['sim_time'] = ds.time ds = ds.drop('time').rename({'time': 'ntime'}) return ds ``` However, this creates a dataset structure which is less useful in my case: ![new_way](https://user-images.githubusercontent.com/761677/196386463-14485d5e-5da6-4443-8a15-4150b65633d8.png) Here, time is lost as a coordinate, and I have to explicitly work the new ""sim_time"" variable to make sense of my data variables. This is not impossible to work with, but I preferred the previous output. My questions are: 1) is the change in behavior in xarray as intended and 2) is there a way for me to get the old dataset structure out of a new version of xarray? ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7180/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 258735262,MDU6SXNzdWUyNTg3MzUyNjI=,1578,Contourplot not producing plot when data below colorbar level and extend='neither',761677,open,0,,,7,2017-09-19T08:23:14Z,2019-03-29T22:44:15Z,,NONE,,,,"I'm plotting several datasets on a map with contourf with the same colorbar levels where some datasets have no data above the lowest level. I expect these plots to simply be blank. The default behavior is instead that 'extend' becomes set to 'min' behind the scenes, changing the colorabar relative to the other plots. When I set extend to 'neither' explicitly, I get an error from matplotlib: ""ValueError: 'bboxes' cannot be empty"". The plot call looks like this: `data.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), vmin=levels[0], vmax=levels[-1], levels=levels, cmap=plt.get_cmap('viridis_r'), extend='neither', extent=(-17, 25, 52, 73)` I found that if I revert this bug-fixe from Cartopy: https://github.com/SciTools/cartopy/issues/811 I get the expected behavior. Not sure if this issue should go to Cartopy but I'm starting here. Matplotlib version 2.0.0 Xarray version 0.9.6 Cartopy version 0.15.1 ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1578/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 255897797,MDU6SXNzdWUyNTU4OTc3OTc=,1559,Subsetting not reducing memory consumption with mfdataset on Windows,761677,closed,0,,,5,2017-09-07T10:58:30Z,2017-09-14T09:51:51Z,2017-09-14T09:51:51Z,NONE,,,,"On Windows, with the following code I run out of memory: ``` d = xr.open_mfdataset(filepath) sed = d.sediment_mass_per_unit_area.isel(time=-1) sed.load() d.close() ``` but when using ""open_dataset"" I load the data instantly: ``` d = xr.open_dataset(filepath) sed = d.sediment_mass_per_unit_area.isel(time=-1) sed.load() d.close() ``` On linux both methods use minimal memory. The target file is 1.5 GB, and the target variable is 1460x862x900 floats. Filepath is a single file, not multiple files. Xarray 0.9.6 used on both systems, installed with conda.","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1559/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue