home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 1706179211

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1706179211 I_kwDOAMm_X85lsjqL 7837 Weighted reductions inconsistency when variables have missing dimensions 22245117 open 0     0 2023-05-11T16:40:58Z 2023-11-06T06:01:35Z   CONTRIBUTOR      

What happened?

There is some inconsistencies in the error raised by weighted reductions when the dimensions over which to apply the reduction are not present in all variables.

What did you expect to happen?

I think all reduction methods should have the same behaviour. I'm not sure what's the best behaviour, although I probably prefer the mean behaviour (do not raise an error when there are variables with missing dimensions).

Minimal Complete Verifiable Example

```Python import xarray as xr

ds = xr.Dataset({"foo": xr.DataArray([[1]] * 2), "bar": 1}) weighted = ds.weighted(ds["dim_0"])

weighted.mean(ds.dims) # OK weighted.std(ds.dims) # ValueError ```

MVCE confirmation

  • [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • [X] Complete example — the example is self-contained, including all data and the text of any traceback.
  • [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python

ValueError Traceback (most recent call last) Cell In[1], line 7 4 weighted = ds.weighted(ds["dim_0"]) 6 weighted.mean(ds.dims) # OK ----> 7 weighted.std(ds.dims) # ValueError

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:503, in Weighted.std(self, dim, skipna, keep_attrs) 497 def std( 498 self, 499 dim: Dims = None, 500 skipna: bool | None = None, 501 keep_attrs: bool | None = None, 502 ) -> T_Xarray: --> 503 return self._implementation( 504 self._weighted_std, dim=dim, skipna=skipna, keep_attrs=keep_attrs 505 )

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:540, in DatasetWeighted._implementation(self, func, dim, kwargs) 537 def _implementation(self, func, dim, kwargs) -> Dataset: 538 self._check_dim(dim) --> 540 return self.obj.map(func, dim=dim, **kwargs)

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/dataset.py:5964, in Dataset.map(self, func, keep_attrs, args, kwargs) 5962 if keep_attrs is None: 5963 keep_attrs = _get_keep_attrs(default=False) -> 5964 variables = { 5965 k: maybe_wrap_array(v, func(v, *args, kwargs)) 5966 for k, v in self.data_vars.items() 5967 } 5968 if keep_attrs: 5969 for k, v in variables.items():

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/dataset.py:5965, in <dictcomp>(.0) 5962 if keep_attrs is None: 5963 keep_attrs = _get_keep_attrs(default=False) 5964 variables = { -> 5965 k: maybe_wrap_array(v, func(v, args, *kwargs)) 5966 for k, v in self.data_vars.items() 5967 } 5968 if keep_attrs: 5969 for k, v in variables.items():

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:309, in Weighted._weighted_std(self, da, dim, skipna) 301 def _weighted_std( 302 self, 303 da: DataArray, 304 dim: Dims = None, 305 skipna: bool | None = None, 306 ) -> DataArray: 307 """Reduce a DataArray by a weighted std along some dimension(s).""" --> 309 return cast("DataArray", np.sqrt(self._weighted_var(da, dim, skipna)))

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:295, in Weighted._weighted_var(self, da, dim, skipna) 287 def _weighted_var( 288 self, 289 da: DataArray, 290 dim: Dims = None, 291 skipna: bool | None = None, 292 ) -> DataArray: 293 """Reduce a DataArray by a weighted var along some dimension(s).""" --> 295 sum_of_squares = self._sum_of_squares(da, dim=dim, skipna=skipna) 297 sum_of_weights = self._sum_of_weights(da, dim=dim) 299 return sum_of_squares / sum_of_weights

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:259, in Weighted._sum_of_squares(self, da, dim, skipna) 251 def _sum_of_squares( 252 self, 253 da: DataArray, 254 dim: Dims = None, 255 skipna: bool | None = None, 256 ) -> DataArray: 257 """Reduce a DataArray by a weighted sum_of_squares along some dimension(s).""" --> 259 demeaned = da - da.weighted(self.weights).mean(dim=dim) 261 return self._reduce((demeaned**2), self.weights, dim=dim, skipna=skipna)

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:483, in Weighted.mean(self, dim, skipna, keep_attrs) 477 def mean( 478 self, 479 dim: Dims = None, 480 skipna: bool | None = None, 481 keep_attrs: bool | None = None, 482 ) -> T_Xarray: --> 483 return self._implementation( 484 self._weighted_mean, dim=dim, skipna=skipna, keep_attrs=keep_attrs 485 )

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:529, in DataArrayWeighted._implementation(self, func, dim, kwargs) 528 def _implementation(self, func, dim, kwargs) -> DataArray: --> 529 self._check_dim(dim) 531 dataset = self.obj._to_temp_dataset() 532 dataset = dataset.map(func, dim=dim, **kwargs)

File ~/mambaforge/envs/xarray/lib/python3.11/site-packages/xarray/core/weighted.py:203, in Weighted._check_dim(self, dim) 201 missing_dims = set(dims) - set(self.obj.dims) - set(self.weights.dims) 202 if missing_dims: --> 203 raise ValueError( 204 f"{self.class.name} does not contain the dimensions: {missing_dims}" 205 )

ValueError: DataArrayWeighted does not contain the dimensions: {'dim_1'} ```

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS ------------------ commit: None python: 3.11.3 | packaged by conda-forge | (main, Apr 6 2023, 09:05:00) [Clang 14.0.6 ] python-bits: 64 OS: Darwin OS-release: 22.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: None libnetcdf: None xarray: 2023.4.2 pandas: 2.0.1 numpy: 1.24.3 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 67.7.2 pip: 23.1.2 conda: None pytest: None mypy: None IPython: 8.13.2 sphinx: None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7837/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 0 rows from issue in issue_comments
Powered by Datasette · Queries took 0.698ms · About: xarray-datasette