Throwing this out there - happy to be shot down if people are opposed.

Current behavior / griping

Currently, coercing a dataset to a boolean invokes ds.__bool__, which in turn calls bool(ds.data_vars):

python class Dataset(DataWithCoords, DatasetArithmetic, Mapping): ... def __bool__(self) -> bool: return bool(self.data_vars)

This has the unfortunate property of returning True as long as there is at least one data_variable, regardless of the contents.

Currently, the behavior of Dataset.__bool__ is, at least as far as I've seen, never helpful but frequently unhelpful. I've seen (and written) tests written for DataArrays being passed a Dataset and suddenly the tests are meaningless so many times. Conversely, I've never found a legitimate use case for bool(ds). As far as I can tell, this is essentially the same as len(ds.data_vars) > 0.

In fact, while testing out my proposed changes below on a fork, I found two tests in the xarray test suite that had succumbed to this issue: see https://github.com/pydata/xarray/pull/6122 and https://github.com/pydata/xarray/pull/6123.

This has been discussed before - see https://github.com/pydata/xarray/issues/4290. This discussion focused on the question "should bool(xr.Dataset({'a': False})) return False?". I agree that it's not clear when it should be false and picking a behavior which deviates from Mapping feels arbitrary and gross.

Proposed behavior

I'm proposing that the API be changed, so that bool(xr.Dataset({'a': False})) raise an error, similar to the implementation in pd.Series.

In this implementation in pandas, attempting to evaluate even a single-element series as a boolean raises an error:

```python In [14]: bool(pd.Series([False]))

ValueError Traceback (most recent call last) <ipython-input-14-b0ad7f4d9277> in <module> ----> 1 bool(pd.Series([False]))

~/miniconda3/envs/rhodium-env/lib/python3.9/site-packages/pandas/core/generic.py in nonzero(self) 1532 @final 1533 def nonzero(self): -> 1534 raise ValueError( 1535 f"The truth value of a {type(self).name} is ambiguous. " 1536 "Use a.empty, a.bool(), a.item(), a.any() or a.all()."

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). ```

I understand hesitancy around changing the core API. That said, if anyone can find an important, correct use of bool(ds) in the wild I'll eat my hat :)

Implementation

This could be as simple as raising an error on ds.__bool__, something like:

python class Dataset(DataWithCoords, DatasetArithmetic, Mapping): ... def __bool__(self) -> bool: raise ValueError( "The truth value of a Dataset is ambiguous. Reduce the data " "variables to a scalar value with any(ds.values()) or " "all(ds.values())." )

The only other change that would be needed is an assertion that directly calls bool(ds) in test_dataset::TestDataset.test_properties, which checks for the exact behavior I'm changing:

python assert bool(ds)

This would need to be changed to:

python with pytest.raises(ValueError): bool(ds)

If this sounds good, I can submit a PR with these changes.

The structure of the Dataset.to_zarr encoding argument is particular to xarray (at least, it's not immediately obvious from the zarr docs how this argument gets parsed) and it took a bit of trial and error to figure out out the rules. Hoping this docs block is helpful to others!

Documentation additions only (no workflow stages)

What is your issue?

I'm pretty sure my PR https://github.com/pydata/xarray/pull/6542 is the culprit. I never figured out how to get around the build timeout with these docs edits.

If you all are on top of this then no worries - feel free to close. Just wanted to point in the right direction so you don't need to go hunting.

• [x] Closes #6466
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst - not sure if this is warranted or if it should go in new features or bug fixes. happy to change/remove.

Is your feature request related to a problem?

xr.Dataset.where and xr.DataArray.where currently do not allow providing both other and drop=True, stating that they are mutually exclusive. Conceptually, this doesn't strike me as true. Drop does not drop all points where cond is False, only those where an entire coordinate label evaluates to False. This is most important when trying to avoid type promotion, such as in the below example, where an integer FillValue might be preferred over dtypes.NA.

```python In [2]: da = xr.DataArray(np.arange(16).reshape(4, 4), dims=['x', 'y'])

In [3]: da Out[3]: <xarray.DataArray (x: 4, y: 4)> array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) Dimensions without coordinates: x, y If *`other`* is not provided, the array is promoted to `float64` to accommodate the default,`dtypes.NA`:python In [4]: da.where(da > 6, drop=True) Out[4]: <xarray.DataArray (x: 3, y: 4)> array([[nan, nan, nan, 7.], [ 8., 9., 10., 11.], [12., 13., 14., 15.]]) Dimensions without coordinates: x, y However, the combination of *`other`* and *`drop=True`* is not allowed:python In [5]: da.where(da > 6, -1, drop=True)

ValueError Traceback (most recent call last) Input In [5], in <module> ----> 1 da.where(da > 6, -1, drop=True)

File ~/miniconda3/envs/rhodium-env/lib/python3.10/site-packages/xarray/core/common.py:1268, in DataWithCoords.where(self, cond, other, drop) 1266 if drop: 1267 if other is not dtypes.NA: -> 1268 raise ValueError("cannot set other if drop=True") 1270 if not isinstance(cond, (Dataset, DataArray)): 1271 raise TypeError( 1272 f"cond argument is {cond!r} but must be a {Dataset!r} or {DataArray!r}" 1273 )

ValueError: cannot set other if drop=True ```

Describe the solution you'd like

Current implementation

The current behavior is enforced within the block handling the drop argument (currently https://github.com/pydata/xarray/blob/main/xarray/core/common.py#L1266-L1268):

python if drop: if other is not dtypes.NA: raise ValueError("cannot set `other` if drop=True")

Proposed fix

I just removed the above if statement on a fork, and the example now works!

```python

import xarray as xr, numpy as np da = xr.DataArray(np.arange(16).reshape(4, 4), dims=['x', 'y']) da <xarray.DataArray (x: 4, y: 4)> array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) Dimensions without coordinates: x, y da.where(da > 6, drop=True) <xarray.DataArray (x: 3, y: 4)> array([[nan, nan, nan, 7.], [ 8., 9., 10., 11.], [12., 13., 14., 15.]]) Dimensions without coordinates: x, y da.where(da > 6, -1, drop=True) <xarray.DataArray (x: 3, y: 4)> array([[-1, -1, -1, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) Dimensions without coordinates: x, y ``` Making this change fails a single test in https://github.com/pydata/xarray/blob/main/xarray/tests/test_dataset.py#L4548-L4549, which explicitly checks for this behavior:

python with pytest.raises(ValueError, match=r"cannot set"): ds.where(ds > 1, other=0, drop=True) This could easily be reworked to check for valid handling of both arguments.

Describe alternatives you've considered

No response

Additional context

I haven't yet investigated what would happen with chunked, sparse, or other complex arrays, or if it's compatible with trees and other things on the roadmap. It's possible this breaks things I'm not imagining. Currently, where(cond, other) and where(cond, drop=True) are well-tested, flexible operations, and I don't see why allowing their union would break anything, but I'll wait to hear from the experts on that front!

I'm definitely open to creating a pull request (and have the simple implementation I've outlined here ready to go).

• [x] Closes #5994
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

@Skealz's great issue (#5994) highlighted that DataArrays with a single non-null value raise an error if DataArray.interpolate_na is called with method='nearest'. Note that this does not occur for the default (method='linear').

In this PR: * Uses the "fast track" to quickly return any 1-D slices with a single non-null value in addition to the previous all-nan or no-nan fasttrack * Explicitly check this behavior for all 1-D interpolation methods using parametrized tests * consolidate other fast-track tests (interpolate_allnans and interpolate_nonans) into a single parameterized test

For discussion: * This patch may break existing workflows if they rely on the current behavior, which errors for slices with only a single value

Issues a PendingDeprecationWarning on Dataset.__bool__

• [x] Closes #6124
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst <-- could definitely use a review

a number of assert statements in test_dataset.py::test_clip make assertions which will never fail as long as there is at least one data_variable:

python assert result.min(...) >= 0.5 this evaluates to datasets with scalar True or False values in each data_variable; however, ds.__bool__ returns true if len(ds.data_vars) > 0.

related: https://github.com/pydata/xarray/pull/6122

No workflow stages here - I just made the changes in the browser. Pretty simple patch that increases the effectiveness of existing tests - no changes to the xarray core code. Let me know if you'd like me to implement any of the normal checks.

Makes four changes to xarray/tests/test_backends.py::test_open_fsspec

Previously, there were four inequality tests which will always pass regardless of whether the read was successful.

assert ds0 == ds2 will always evaluate to True if the datasets have at least one data variable. Instead, xr.testing.assert_equal should be used to test data variable equality.

No workflow stages here - I just made the changes in the browser. Pretty simple patch that increases the effectiveness of existing tests - no changes to the xarray core code.

If helpful - added a PR to adds a reference and link to xr.set_options in the section of the FAQ on "what is your approach to metadata?". I had to do a bit of digging to figure out what was meant by "there is a global flag" and thought others might find the direct ref useful.

Relevant section: https://github.com/pydata/xarray/blob/main/doc/getting-started-guide/faq.rst#what-is-your-approach-to-metadata

Current text:

An implication of this choice is that we do not propagate attrs through most operations unless explicitly flagged (some methods have a keep_attrs option, and there is a global flag for setting this to be always True or False)

Proposed version:

An implication of this choice is that we do not propagate attrs through most operations unless explicitly flagged (some methods have a keep_attrs option, and there is a global flag, accessible with xr.set_options, for setting this to be always True or False).

Minor docs change only. No workflow steps needed.

very simple fix in the broadcasting by dimension name docs:

DataArray objects ~~are~~ automatically align themselves [...] by dimension name instead of axis order.

No workflow stages - this is a very minor docs fix.

The use case is for netCDF files stored on s3 or other generic cloud storage

```python import requests, xarray as xr fp = 'http://nasanex.s3.amazonaws.com/NEX-GDDP/BCSD/rcp45/day/atmos/tasmax/r1i1p1/v1.0/tasmax_day_BCSD_rcp45_r1i1p1_MPI-ESM-LR_2029.nc'

data = requests.get(fp, stream=True) ds = xr.open_dataset(data.content) # raises TypeError: embedded NUL character ```

Ideal would be integration with the (hopefully) soon-to-be implemented dask.distributed features discussed in #798.

series.to_xarray() fails when MultiIndex not sorted in xarray 0.15.1

Summary

It seems that series.to_xarray() fails (returns incorrect data) in xarray 0.15.1 when the dataframe's MultiIndex dimensions are not sorted

Demonstration

xarray should be able to handle MultiIndices with unsorted dimensions. Using a fresh conda environment with xarray 0.14.1:

```python $ conda run -n py37xr14 python test.py

df alpha B A num 0 1 4 1 2 5 2 3 6

df.stack('alpha') num alpha 0 B 1 A 4 1 B 2 A 5 2 B 3 A 6 dtype: int64

df.stack('alpha').to_xarray() <xarray.DataArray (num: 3, alpha: 2)> array([[1, 4], [2, 5], [3, 6]]) Coordinates: * num (num) int64 0 1 2 * alpha (alpha) object 'B' 'A' ```

This fails in xarray 0.15.1 - note the data is not merely reordered - the data in column 'B' now has the incorrect values 4, 5, 6 rather than 1, 2, 3:

```python $ conda run -n py37xr15 python test.py

df alpha B A num 0 1 4 1 2 5 2 3 6

df.stack('alpha') num alpha 0 B 1 A 4 1 B 2 A 5 2 B 3 A 6 dtype: int64

df.stack('alpha').to_xarray() <xarray.DataArray (num: 3, alpha: 2)> array([[4, 1], [5, 2], [6, 3]]) Coordinates: * num (num) int64 0 1 2 * alpha (alpha) object 'B' 'A' ```

Test setup & environment info

Simple spelling correction.

Add space to end of line halfway through error message on line 682 to avoid 'bebroadcast' on string concatenation.

• [x] Closes #2071
• ~~Tests added (for all bug fixes or enhancements)~~
• [ ] Tests passed (for all non-documentation changes)
• ~~Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)~~

Spelling error in value error raised on index-based assignment with incorrect shape

Very easy one here:

```python In [1]: import xarray as xr, pandas as pd, numpy as np

In [2]: da = xr.DataArray(np.random.random((2, 3)), dims=['x','y'])

In [3]: da Out[3]: <xarray.DataArray (x: 2, y: 3)> array([[0.882927, 0.604024, 0.316146], [0.06342 , 0.503182, 0.297988]]) Dimensions without coordinates: x, y

In [4]: da[0, 1] = [1, 2]

ValueError Traceback (most recent call last) <ipython-input-4-1fbe1d206e00> in <module>() ----> 1 da[0, 1] = [1, 2]

~/miniconda2/envs/xarray-dev/lib/python3.6/site-packages/xarray/core/dataarray.py in setitem(self, key, value) 486 key = {k: v.variable if isinstance(v, DataArray) else v 487 for k, v in self._item_key_to_dict(key).items()} --> 488 self.variable[key] = value 489 490 def delitem(self, key):

~/miniconda2/envs/xarray-dev/lib/python3.6/site-packages/xarray/core/variable.py in setitem(self, key, value) 682 'shape mismatch: value array of shape %s could not be' 683 'broadcast to indexing result with %s dimensions' --> 684 % (value.shape, len(dims))) 685 if value.ndim == 0: 686 value = Variable((), value)

ValueError: shape mismatch: value array of shape (2,) could not bebroadcast to indexing result with 0 dimensions

In [5]: xr.show_versions() /Users/delgadom/miniconda2/envs/xarray-dev/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters

INSTALLED VERSIONS

commit: None python: 3.6.4.final.0 python-bits: 64 OS: Darwin OS-release: 17.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

xarray: 0.10.2 pandas: 0.22.0 numpy: 1.14.2 scipy: 1.0.0 netCDF4: 1.3.1 h5netcdf: 0.5.0 h5py: 2.7.1 Nio: None zarr: 2.2.0 bottleneck: 1.2.1 cyordereddict: None dask: 0.17.2 distributed: 1.21.4 matplotlib: 2.2.2 cartopy: 0.15.1 seaborn: 0.8.1 setuptools: 38.5.1 pip: 9.0.1 conda: None pytest: 3.4.1 IPython: 6.2.1 sphinx: 1.6.6

```

Problem description

The error message in variable.py#L682 seems to be missing an end-of-line space. Happy to create a PR.

• [x] Closes #1781
• [x] Tests added
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

For testing - I could use some guidance. Not sure if it's worth creating a fixture set or something just for this issue. If so, would that go in test_backends?

Using both xarray 0.9.6 and current (0.9.6-16-gb201ff7), da.sel(drop=True, **indexers) and da.isel(drop=True, **indexers) return unexpectedly different results.

Setup:

```python In [1]: import xarray as xr, pandas as pd, numpy as np

In [2]: years = pd.Index( ...: pd.date_range('1981-01-01', '2100-01-01', freq='A', closed='left'), ...: name='time') ...: ages = pd.Index(['age0', 'age1', 'age2', 'age3'], name='age')

In [3]: arr = xr.DataArray( ...: np.random.random((len(years), 4)), dims=('time', 'age'), ...: coords={'time': years, 'age': ages})

In [4]: arr Out[4]: <xarray.DataArray (time: 119, age: 4)> array([[ 0.755194, 0.1316 , 0.283485, 0.616929], [ 0.01667 , 0.907853, 0.667366, 0.146755], [ 0.338319, 0.782972, 0.367624, 0.390907], ..., [ 0.453521, 0.807693, 0.094811, 0.603297], [ 0.405114, 0.821691, 0.633314, 0.259406], [ 0.41722 , 0.012957, 0.329089, 0.774966]]) Coordinates: * age (age) object 'age0' 'age1' 'age2' 'age3' * time (time) datetime64[ns] 1981-12-31 1982-12-31 1983-12-31 ... ```

I would expect the following operations to return identical results:

```python In [5]: arr.sel(time='2012', drop=True) Out[5]: <xarray.DataArray (time: 1, age: 4)> array([[ 0.086045, 0.467905, 0.101005, 0.503311]]) Coordinates: * age (age) object 'age0' 'age1' 'age2' 'age3' Dimensions without coordinates: time

In [6]: arr.isel(time=31, drop=True) Out[6]: <xarray.DataArray (age: 4)> array([ 0.086045, 0.467905, 0.101005, 0.503311]) Coordinates: * age (age) object 'age0' 'age1' 'age2' 'age3' `` Note thattimeis still seen as a dimension in in the.sel` results.

The same behavior is seen for Dataset.sel(drop=true):

```python In [7]: ds = xr.Dataset({'arr': arr})

In [8]: ds.sel(time='2012', drop=True) Out[8]: <xarray.Dataset> Dimensions: (age: 4, time: 1) Coordinates: * age (age) object 'age0' 'age1' 'age2' 'age3' Dimensions without coordinates: time Data variables: arr (time, age) float64 0.08604 0.4679 0.101 0.5033

In [9]: ds.isel(time=31, drop=True) Out[9]: <xarray.Dataset> Dimensions: (age: 4) Coordinates: * age (age) object 'age0' 'age1' 'age2' 'age3' Data variables: arr (age) float64 0.08604 0.4679 0.101 0.5033 ```