issues: 1758106607

What is your issue?

This is a mixture between question, bug (potentially) and general issue, so feel free to label it accordingly.

Here is my question: what is the recommended approach to add a xr.DataArray to an existing xr.Dataset with a MultiIndex?

To give some more context, I've a xarray.Dataset called market with several variables and coordinates, one of them, timeslice, a MultiIndex. This is what it looks like:

<xarray.Dataset> Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0

Now, I want to add another variable, called supply, identical to exports but filled with zeros. In a code that was working with xarray==2022.3.0 and pandas==1.4.4, I was simply doing:

python market["supply"] = xr.zeros_like(market.exports)

And it worked totally fine. With the newest versions of xarray==2023.5.0 and pandas==2.0.2 under python 3.10, this fails with:

*** DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'timeslice'} to avoid an error in the future. I've tried variants like:

python market["supply"] = market.exports * 0 market = market.assign(supply = zeros_like(market.exports)) both failing with the same message.

I totally fail to see how this process is deleting a level of the MultiIndex - or modifying the indexes in any form. Probably it is because I don't understand the inner workings of xarray indexes.

The following works totally fine, but it is rather convoluted having to create a brand new Dataset from scratch manually, in addition to be problematic if you really want to modify the Dataset in place (same problem will have assign).

python vars = dict(market.data_vars) vars["supply"] = xr.zeros_like(market.exports) market = xr.Dataset(vars)

Resulting in:

<xarray.Dataset> Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 supply (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0

Many thanks for your support!