issues: 1329754426

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1329754426 I_kwDOAMm_X85PQnE6 6879 `Dataset.where()` incorrectly applies mask and creates new dimensions 12760310 closed 0     3 2022-08-05T10:30:41Z 2022-08-05T12:10:33Z 2022-08-05T12:10:33Z NONE      

What happened?

Suppose I have the following dataset

python <xarray.Dataset> Dimensions: (lat: 468, lon: 520, n_stations_t_2m_min_anom: 930) Coordinates: * lon (lon) float64 6.012 6.037 6.062 ... 18.96 18.99 * lat (lat) float64 36.01 36.04 36.06 ... 47.66 47.69 Dimensions without coordinates: n_stations_t_2m_min_anom Data variables: t_2m_min_anom (lat, lon) float32 ... t_2m_min_anom_stations (n_stations_t_2m_min_anom) float64 1.935 ... 0.8557

and a mask to apply

python <xarray.DataArray 'mask' (lat: 468, lon: 520)> array([[False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], ..., [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False]]) Coordinates: * lat (lat) float64 36.01 36.04 36.06 36.09 ... 47.61 47.64 47.66 47.69 * lon (lon) float64 6.012 6.037 6.062 6.087 ... 18.91 18.94 18.96 18.99

If I apply the mask to the dataset doing

python data = data.where(mask)

then lat, lon dimensions are also broadcasted to t_2m_min_anom_stations, which is a 1-D array that does NOT have these coordinates.

python <xarray.Dataset> Dimensions: (lat: 468, lon: 520, n_stations_t_2m_min_anom: 930) Coordinates: * lon (lon) float64 6.012 6.037 6.062 ... 18.96 18.99 * lat (lat) float64 36.01 36.04 36.06 ... 47.66 47.69 region int64 0 abbrevs <U2 'r0' names <U7 'Region0' Dimensions without coordinates: n_stations_t_2m_min_anom Data variables: t_2m_min_anom (lat, lon) float32 nan nan nan nan ... nan nan nan t_2m_min_anom_stations (n_stations_t_2m_min_anom, lat, lon) float64 nan ...

This causes all sort of issues as the newly created array t_2m_min_anom_stations is too large to be hold in memory

What did you expect to happen?

The final result should apply mask only on the variables that have lat,lon as dimensions

python <xarray.Dataset> Dimensions: (lat: 468, lon: 520, n_stations_t_2m_min_anom: 930) Coordinates: * lon (lon) float64 6.012 6.037 6.062 ... 18.96 18.99 * lat (lat) float64 36.01 36.04 36.06 ... 47.66 47.69 region int64 0 abbrevs <U2 'r0' names <U7 'Region0' Dimensions without coordinates: n_stations_t_2m_min_anom Data variables: t_2m_min_anom (lat, lon) float32 nan nan nan nan ... nan nan nan t_2m_min_anom_stations (n_stations_t_2m_min_anom) float64 1.935 ... 0.8557

Minimal Complete Verifiable Example

No response

MVCE confirmation

  • [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
  • [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS ------------------ commit: None python: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-229.1.2.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.utf8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.2.3 numpy: 1.20.3 scipy: 1.8.1 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.1 nc_time_axis: None PseudoNetCDF: None rasterio: 1.2.1 cfgrib: None iris: None bottleneck: None dask: 2022.7.1 distributed: 2022.7.1 matplotlib: 3.5.2 cartopy: 0.18.0 seaborn: 0.11.2 numbagg: None fsspec: 2022.5.0 cupy: None pint: 0.19.2 sparse: None setuptools: 59.8.0 pip: 22.2 conda: 4.13.0 pytest: None IPython: 8.4.0 sphinx: None 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/6879/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables