home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

2 rows where issue = 1364911775 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

  • jamesstidard 1
  • benbovy 1

author_association 2

  • MEMBER 1
  • NONE 1

issue 1

  • Cannot re-index or align objects with conflicting indexes · 2 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1240646680 https://github.com/pydata/xarray/issues/7005#issuecomment-1240646680 https://api.github.com/repos/pydata/xarray/issues/7005 IC_kwDOAMm_X85J8sQY jamesstidard 1797906 2022-09-08T12:24:41Z 2022-09-08T12:24:41Z NONE

Hi @benbovy,

Thanks for the detailed response.

Yeah, that it was only raising for the second multi indexing map, does seem like a bug in that case, I'll leave the ticket open to track that.

I didn't stumble on the set_levels function while skimming though the docs, thanks. I've updated my function to make use of that. I'm hoping things should be safe with this, and I'm correctly replacing things in the correct order.

For anyone else who's looking to do the same, or for anyone to tell me what I'm doing is not safe, or there's a simpler way, here's the updated function:

```python import numpy as np import pandas as pd import xarray as xr

def map_coords(ds, *, name, mapping): """ Takes a xarray dataset's coordinate values and updates them with the given the provided mapping. In-place.

Can handle both regular indices and multi-level indices.

ds: An xr.Datset
name: name of the coordinate to update
mapping: dictionary, key of old value, value of new value.
"""
# all attrs seem to get dropped on coords even if only
# one is altered. Hold on to them and reapply after
coord_attrs = {c: ds[c].attrs for c in ds.coords}

if ds.indexes.is_multi(name):
    target = name
    parent = ds.indexes[target].name

    if target == parent:
        valid_targets = ds.indexes[parent].names
        raise ValueError(
            f"Can only map levels of a MultiIndex, not the MultiIndex "
            f"itself. Target one of {valid_targets}",
        )

    multi_index = ds.indexes[parent]
    level_values = dict(zip(multi_index.names, multi_index.levels))
    new_values = [mapping[v] for v in level_values[name]]
    ds.coords[parent] = multi_index.set_levels(new_values, level=target)
else:
    old_values = ds.coords[name].values
    new_values = [mapping[v] for v in old_values]
    ds[name] = new_values

# reapply attrs
for coord, attrs in coord_attrs.items():
    ds[coord].attrs = attrs

midx = pd.MultiIndex.from_product([list("abc"), [0, 1]], names=("x_one", "x_two")) midy = pd.MultiIndex.from_product([list("abc"), [0, 1]], names=("y_one", "y_two")) mda = xr.DataArray(np.random.rand(6, 6, 3), [("x", midx), ("y", midy), ("z", range(3))])

map_coords(mda, name="z", mapping={0: "zero", 1: "one", 2: "two"}) map_coords(mda, name="x_one", mapping={"a": "aa", "b": "bb", "c": "cc"}) map_coords(mda, name="y_one", mapping={"a": "aa", "b": "bb", "c": "cc"})

print(mda) ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot re-index or align objects with conflicting indexes 1364911775
1240532341 https://github.com/pydata/xarray/issues/7005#issuecomment-1240532341 https://api.github.com/repos/pydata/xarray/issues/7005 IC_kwDOAMm_X85J8QV1 benbovy 4160723 2022-09-08T10:32:31Z 2022-09-08T10:32:31Z MEMBER

Hi @jamesstidard,

causes a ValueError I am struggling to interpret.

I agree that the error message is not very informative. This could be improved.

The problem in your example is located here:

python def map_coords(ds, *, name, mapping): ... ds.coords[name] = xr.DataArray(new_values, coords=coord.coords) ...

Since v2022.6.0, multi-index level coordinates are real coordinates. What happens is that when you update "x_one" it invalidates the multi-index "x" - "x_one" - "x_two" in the DataArray:

```python ... map_coords(mda, name="z", mapping={0: "zero", 1: "one", 2: "two"}). # success map_coords(mda, name="x_one", mapping={"a": "aa", "b": "bb", "c": "cc"}) # success

the 'x', 'x_one' and 'x_two' indexes are all equal

mda.xindexes["x_one"].equals(mda.xindexes["x"]) and mda.xindexes["x_one"].equals(mda.xindexes["x_two"])

True

the 'x' and 'x_two' indexes are identical, but not the 'x_one' index!

mda.xindexes["x"] is mda.xindexes["x_two"]

True

mda.xindexes["x"] is mda.xindexes["x_one"]

False

```

Which explains the alignment error when you further try to update the DataArray.

I'm actually surprised that it doesn't raise any error at that point, like this:

```python mda = xr.DataArray(np.random.rand(6, 6, 3), [("x", midx), ("y", midy), ("z", range(3))]) mda.coords["x_two"] = ("x", range(6))

ValueError: cannot set or update variable(s) 'x_two', which would corrupt the following

index built from coordinates 'x', 'x_one', 'x_two':

<xarray.core.indexes.PandasMultiIndex object at 0x1689cc190>

```

It seems like a bug.

Either way, you shouldn't try updating a single coordinate that is part of a multi-coordinate index, which is assumed immutable. Instead, you could use reset_index/set_index before/after updating the labels of one level coordinate. You could also replace the whole index and its coordinates at once (example below). I wouldn't rely too much on the latter option, though, as such special case for pd.MultiIndex might eventually be deprecated in Xarray.

python new_midx = midx.set_levels(["aa", "bb", "cc"], level="x_one") mda.coords["x"] = new_midx

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Cannot re-index or align objects with conflicting indexes 1364911775

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 13.613ms · About: xarray-datasette