home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 2095994466

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
2095994466 I_kwDOAMm_X8587lZi 8646 `rename_vars` followed by `swap_dims` and `merge` causes swapped dim to reappear 11130776 open 0     16 2024-01-23T12:31:54Z 2024-04-05T09:10:15Z   NONE      

What happened?

I wanted to rename a dimension coordinate for two datasets before merging: ds = ds.rename_vars(y="z").swap_dims(y="z"), and the same for the second data set. After merging the datasets, the merged result has the dimension "y" in addition to "z".

Swapping the order of rename_vars and swap_dims before merging works in that "y" does not reappear, but then "z" is listed as a non-dimension coordinate.

Doing rename_vars followed by swap_dims /after/ merging gives the result I wanted, but if I merge again, the same issue occurs.

My current solution is to only rename dimension coordinates before saving to netCDF.

What did you expect to happen?

Merging two datasets with the same coordinates and dimensions (but different data variables) should result in a single dataset with all of the data variables from the two datasets and exactly the same coordinates and dimensions.

Minimal Complete Verifiable Example

```Python import numpy as np import xarray as xr from xarray.core.utils import Frozen

A = np.arange(4).reshape((2, 2)) B = np.arange(4).reshape((2, 2)) + 4 ds1 = xr.Dataset({"A": (["x", "y"], A), "B": (["x", "y"], B)}, coords={"x": ("x", [1, 2]), "y": ("y", [1, 2])}) ds2 = xr.Dataset({"C": (["x", "y"], A), "D": (["x", "y"], B)}, coords={"x": ("x", [1, 2]), "y": ("y", [1, 2])})

assert ds1.dims == Frozen({"x": 2, "y": 2}) assert ds2.dims == Frozen({"x": 2, "y": 2})

ds1_swap = ds1.rename_vars(y="z").swap_dims(y="z") ds2_swap = ds2.rename_vars(y="z").swap_dims(y="z")

assert ds1_swap.dims == Frozen({"x": 2, "z": 2}) assert ds2_swap.dims == Frozen({"x": 2, "z": 2})

merging makes the dimension "y" reappear (I would expect this assertion to fail):

assert xr.merge([ds1_swap, ds2_swap]).dims == Frozen({"x": 2, "z": 2, "y": 2})

renaming and swapping after the merge causes issues later:

ds12 = xr.merge([ds1, ds2]).rename_vars(y="z").swap_dims(y="z") ds3 = xr.Dataset({"E": (["x", "z"], A), "F": (["x", "z"], B)}, coords={"x": ("x", [1, 2]), "z": ("z", [1, 2])})

ds12 and ds3 have the same dimensions:

assert ds12.dims == Frozen({"x": 2, "z": 2}) assert ds3.dims == Frozen({"x": 2, "z": 2})

but merging brings back "y"

ds123 = xr.merge([ds12, ds3]) assert ds123.dims == Frozen({"x": 2, "z": 2, "y": 2})

as do other operations:

ds12_as = ds12.assign_coords(x=(ds12.x + 1)) assert ds12_as.sizes == Frozen({"x": 2, "z": 2, "y": 2}) ```

MVCE confirmation

  • [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • [X] Complete example — the example is self-contained, including all data and the text of any traceback.
  • [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
  • [x] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

No response

Anything else we need to know?

No response

Environment

The MVCE works in all venvs I've tried including:

INSTALLED VERSIONS ------------------ commit: None python: 3.10.13 (main, Nov 10 2023, 15:02:19) [GCC 11.4.0] python-bits: 64 OS: Linux OS-release: 6.5.0-14-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: ('en_GB', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.9.3-development xarray: 2023.11.0 pandas: 1.5.3 numpy: 1.26.2 scipy: 1.11.4 netCDF4: 1.6.5 pydap: None h5netcdf: 1.3.0 h5py: 3.10.0 Nio: None zarr: None cftime: 1.6.3 nc_time_axis: 1.4.1 iris: None bottleneck: None dask: 2023.12.0 distributed: None matplotlib: 3.8.2 cartopy: 0.22.0 seaborn: 0.13.0 numbagg: None fsspec: 2023.12.1 cupy: None pint: None sparse: 0.15.1 flox: None numpy_groupies: None setuptools: 69.0.2 pip: 23.3.1 conda: None pytest: 7.4.3 mypy: None IPython: 8.18.1 sphinx: None /home/brendan/Documents/inversions/.pymc_venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.")
INSTALLED VERSIONS ------------------ commit: None python: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.81.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: ('en_GB', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2024.1.0 pandas: 2.2.0 numpy: 1.26.3 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 69.0.3 pip: 23.3.2 conda: None pytest: None mypy: None IPython: 8.18.1 sphinx: None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/8646/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 0 rows from issue in issue_comments
Powered by Datasette · Queries took 0.519ms · About: xarray-datasette