issues

Table actions

2 rows where user = 6095790 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 1

state 1

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1758106607 I_kwDOAMm_X85oypPv 7921 Error when adding a DataArray to an existing Dataset with a MultiIndex dalonsoa 6095790 closed 0     10 2023-06-15T06:08:17Z 2023-09-06T06:25:45Z 2023-08-29T14:23:32Z NONE      

What is your issue?

This is a mixture between question, bug (potentially) and general issue, so feel free to label it accordingly.

Here is my question: what is the recommended approach to add a xr.DataArray to an existing xr.Dataset with a MultiIndex?

To give some more context, I've a xarray.Dataset called market with several variables and coordinates, one of them, timeslice, a MultiIndex. This is what it looks like:

<xarray.Dataset> Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0

Now, I want to add another variable, called supply, identical to exports but filled with zeros. In a code that was working with xarray==2022.3.0 and pandas==1.4.4, I was simply doing:

python market["supply"] = xr.zeros_like(market.exports)

And it worked totally fine. With the newest versions of xarray==2023.5.0 and pandas==2.0.2 under python 3.10, this fails with:

*** DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'timeslice'} to avoid an error in the future. I've tried variants like:

python market["supply"] = market.exports * 0 market = market.assign(supply = zeros_like(market.exports)) both failing with the same message.

I totally fail to see how this process is deleting a level of the MultiIndex - or modifying the indexes in any form. Probably it is because I don't understand the inner workings of xarray indexes.

The following works totally fine, but it is rather convoluted having to create a brand new Dataset from scratch manually, in addition to be problematic if you really want to modify the Dataset in place (same problem will have assign).

python vars = dict(market.data_vars) vars["supply"] = xr.zeros_like(market.exports) market = xr.Dataset(vars)

Resulting in:

<xarray.Dataset> Dimensions: (region: 1, commodity: 6, timeslice: 6, year: 8) Coordinates: * region (region) object 'R1' * commodity (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind' units_prices (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt' * timeslice (timeslice) object MultiIndex * month (timeslice) object 'all-year' 'all-year' ... 'all-year' * day (timeslice) object 'all-week' 'all-week' ... 'all-week' * hour (timeslice) object 'night' 'morning' ... 'late-peak' 'evening' * year (year) int64 2020 2025 2030 2035 2040 2045 2050 2055 Data variables: prices (commodity, region, year, timeslice) float64 0.0702 ... 0.0 exports (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0 imports (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 static_trade (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0 supply (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0

Many thanks for your support!

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7921/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed xarray 13221727 issue
425905002 MDU6SXNzdWU0MjU5MDUwMDI= 2854 Assign values to DataArray after interpolating Dataset produces an error dalonsoa 6095790 closed 0     2 2019-03-27T11:27:46Z 2019-04-04T21:58:24Z 2019-04-04T21:58:24Z NONE      

Code Sample, a copy-pastable example if possible

```python import numpy as np import xarray as xr

da = xr.DataArray( np.random.random((3, 4)), dims=["assets", "year"], coords={"year": range(2010, 2018, 2)} )

Interpolate directly the DataArray

new_da = da.interp(year=[2011, 2015]) new_da.loc[{"year": 2015}] = 42 # WORKS! print(new_da)

Interpolate DataArray as part of a Dataset

dset = xr.Dataset({"my_array": da.copy()}).interp(year=[2011, 2015]) new_da = dset.my_array new_da.loc[{"year": 2015}] = 42 # DOESN'T WORK! print(new_da) ```

Problem description

Assigning a value to some locations of a DataArray after it has been interpolated raises an IndexError. This does not occur when the DataArray is interpolated independently, only occurs when the DataArray has been interpolated as part of a Dataset.

This problem is not present in xarray<0.12.0

Expected Output

The above example should print new_da in both cases, with the values corresponding to year 2015 replaced by 42. That works in the first case, but fails in the second raising IndexError.

Although I am not sure if it is the cause of this error, I have found that in the first case the array indexes are present and contain [2011, 2015], but they are an empty OrderDict in the second case.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.1 (default, Nov 6 2018, 18:46:03) [Clang 10.0.0 (clang-1000.11.45.5)] python-bits: 64 OS: Darwin OS-release: 18.2.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: en_GB.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.4.1.1 xarray: 0.12.0 pandas: 0.23.4 numpy: 1.15.4 scipy: 1.1.0 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nc_time_axis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.2 cartopy: None seaborn: 0.9.0 setuptools: 40.5.0 pip: 18.1 conda: None pytest: 4.0.2 IPython: 7.1.1 sphinx: None 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2854/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);