id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1758106607,I_kwDOAMm_X85oypPv,7921,Error when adding a DataArray to an existing Dataset with a MultiIndex,6095790,closed,0,,,10,2023-06-15T06:08:17Z,2023-09-06T06:25:45Z,2023-08-29T14:23:32Z,NONE,,,,"### What is your issue? This is a mixture between question, bug (potentially) and general issue, so feel free to label it accordingly. Here is my question: **what is the recommended approach to add a `xr.DataArray` to an existing `xr.Dataset` with a `MultiIndex`?** To give some more context, I've a `xarray.Dataset` called `market` with several variables and coordinates, one of them, `timeslice`, a `MultiIndex`. This is what it looks like: ``` Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 ``` Now, I want to add another variable, called `supply`, identical to `exports` but filled with zeros. In a code that was working with `xarray==2022.3.0` and `pandas==1.4.4`, I was simply doing: ```python market[""supply""] = xr.zeros_like(market.exports) ``` And it worked totally fine. With the newest versions of `xarray==2023.5.0` and `pandas==2.0.2` under python 3.10, this fails with: ``` *** DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'timeslice'} to avoid an error in the future. ``` I've tried variants like: ```python market[""supply""] = market.exports * 0 market = market.assign(supply = zeros_like(market.exports)) ``` both failing with the same message. I totally fail to see how this process is deleting a level of the MultiIndex - or modifying the indexes in any form. Probably it is because I don't understand the inner workings of `xarray` indexes. The following works totally fine, but it is rather convoluted having to create a brand new Dataset from scratch manually, in addition to be problematic if you really want to modify the Dataset in place (same problem will have `assign`). ```python vars = dict(market.data_vars) vars[""supply""] = xr.zeros_like(market.exports) market = xr.Dataset(vars) ``` Resulting in: ``` Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 supply (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 ``` Many thanks for your support!","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7921/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 425905002,MDU6SXNzdWU0MjU5MDUwMDI=,2854,Assign values to DataArray after interpolating Dataset produces an error,6095790,closed,0,,,2,2019-03-27T11:27:46Z,2019-04-04T21:58:24Z,2019-04-04T21:58:24Z,NONE,,,,"#### Code Sample, a copy-pastable example if possible ```python import numpy as np import xarray as xr da = xr.DataArray( np.random.random((3, 4)), dims=[""assets"", ""year""], coords={""year"": range(2010, 2018, 2)} ) # Interpolate directly the DataArray new_da = da.interp(year=[2011, 2015]) new_da.loc[{""year"": 2015}] = 42 # WORKS! print(new_da) # Interpolate DataArray as part of a Dataset dset = xr.Dataset({""my_array"": da.copy()}).interp(year=[2011, 2015]) new_da = dset.my_array new_da.loc[{""year"": 2015}] = 42 # DOESN'T WORK! print(new_da) ``` #### Problem description Assigning a value to some locations of a DataArray after it has been interpolated raises an IndexError. This does not occur when the DataArray is interpolated independently, only occurs when the DataArray has been interpolated as part of a Dataset. This problem is not present in xarray<0.12.0 #### Expected Output The above example should print `new_da` in both cases, with the values corresponding to year 2015 replaced by 42. That works in the first case, but fails in the second raising IndexError. Although I am not sure if it is the cause of this error, I have found that in the first case the array indexes are present and contain [2011, 2015], but they are an empty OrderDict in the second case. #### Output of ``xr.show_versions()``
INSTALLED VERSIONS ------------------ commit: None python: 3.7.1 (default, Nov 6 2018, 18:46:03) [Clang 10.0.0 (clang-1000.11.45.5)] python-bits: 64 OS: Darwin OS-release: 18.2.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: en_GB.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.4.1.1 xarray: 0.12.0 pandas: 0.23.4 numpy: 1.15.4 scipy: 1.1.0 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nc_time_axis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.2 cartopy: None seaborn: 0.9.0 setuptools: 40.5.0 pip: 18.1 conda: None pytest: 4.0.2 IPython: 7.1.1 sphinx: None
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2854/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue