Is your feature request related to a problem?

Now that python 3.11 has been released, we should start to support it officially.

Describe the solution you'd like

I guess the first step would be to replace python 3.10 as a maximum version in the tests and see what crashes (and get lucky).

Describe alternatives you've considered

No response

Additional context

No response

• [x] Closes #8546
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Probably we should add a whole bunch of tests for this. For now only testing the constructor.

• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

This PR improves the typing of align. Before: the type of the inputs was reduced to the common superclass and the return type was the same. This often required casts or ignores when mixing classes (e.g. da, ds = xr.align(da, ds). Now: the return types are exactly the same as the input types if the number of passed arguments is <=5.

Only downside: it requires some ugly overloads with type ignores on align. Maybe someone knows how to type this better?

What is your issue?

The doctest is currently failing with

E UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems

• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Mini PR to improve the typing of curvefit. Using dict is problematic since it is invariant, while Mapping is covariant.

• [x] Closes #8448
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

I guess we update manually for now?

What is your issue?

According to the array api standard, the shape property returns tuple[int | None, ...]. Currently we only support tuple[int, ...]

This will actually raise some errors if a duckarray actually returns some None. E.g. NamedArray.size will fail.

(On a side note: dask arrays actually use NaN instead of None for some reason.... Only advantage of this is that NamedArray.size will actually also return NaN instead of raising...)

What happened?

I have a DataArray with a single dimension and multiple (Pandas)Indexes assigned to various coordinates for efficient indexing using sel.

Edit: the problem is even worse than originally described below: such a DataArray breaks all alignment and it's basically unusable...

When I try to add an additional coordinate without any index (I simply use the tuple[dimension, values] way) I get a ValueError about aligning with conflicting indexes.

If the original DataArray only has a single (Pandas)Index everything works as expected.

What did you expect to happen?

I expected that I can simply assign new coordinates without an index.

Minimal Complete Verifiable Example

```Python import xarray as xr

da = xr.DataArray( [1, 2, 3], dims="t", coords={ "a": ("t", [3, 4, 5]), "b": ("t", [5, 6, 7]) } )

set one index

da2 = da.set_xindex("a")

set second index (same dimension, maybe thats a problem?)

da3 = da2.set_xindex("b")

this works

da2.coords["c"] = ("t", [2, 3, 4])

this does not

da3.coords["c"] = ("t", [2, 3, 4]) ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

ValueError: cannot re-index or align objects with conflicting indexes found for the following dimensions: 't' (2 conflicting indexes) Conflicting indexes may occur when - they relate to different sets of coordinate and/or dimension names - they don't have the same type - they may be used to reindex data along common dimensions

Anything else we need to know?

No response

Environment

I have not yet tried this with a newer version of xarray....

• [x] Closes #7802
• [x] Tests added

Mainly deprecated "closed" argument in date_range and passing pd.MultiIndex directly to the constructor.

• [x] Related to #7780
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• [x] Closes #6775
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

• [x] Closes #7478
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Don't know yet how to effectively test this, I guess there is some tricky solution with some mocking.

• [x] Closes #7141

What is your issue?

The coverage was reduced from ~94% to ~68% after merging #7126 See https://app.codecov.io/gh/pydata/xarray or our badge

I think this is because the unittests never included the tests directory while mypy does. And codecov uses the sum of both coverage reports to come up with its number.

Adding the flag to the badge also does not seem to help?

Not sure how or even if that is possible to solve, maybe we need to ask in codecov?

What is your issue?

It seems that the readthedocs build is failing since some upstream update. pydata-sphinx-theme seems to be incompatible with the sphinx-book-theme.

Maybe we have to pin to a specific or a maximum version for now.

What is your issue?

Since cfgrib 0.9.9 (04/2021) it comes with its own xarray backend plugin (looks mainly like a copy of our internal version). We should deprecate our internal plugin.

The deprecation is complicated since we usually bind the minimum version to a minor step, but cfgrib seems to be on 0.9 since 4 years already. Maybye an exception like for netCDF4?

Anyway, if we decide to leave it as it is for now, this ticket is just a reminder to remove it someday :)

• [x] Closes #7479

I don't know how I feel about using private numpy classes that might change anytime. Maybe within a if TYPE_CHECKING block it is not too bad?

• [x] Closes #4975

This little config change will make black remove trailing commas when they are not necessary to fit something into a single line.

It is a pure design choice but personally I like the clean up it does when function signatures simplify (although this happens rarely with more and more type hints added).

I can understand that some people prefer the manual control over what is multiline and what is not. Feel free to vote on it :)

For me it adds cheap LOCs so it looks like I am working hard, haha.

• [x] Closes #7340
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

What is your issue?

In the latest CI runs it seems that test_backends.py::TestNCZarr::test_open_nczarr is failing with

KeyError: 'Zarr object is missing the attribute _ARRAY_DIMENSIONS and the NCZarr metadata, which are required for xarray to determine variable dimensions.'

I don't see an obvious reason for this, especially since the zarr version has not changed compared to some runs that were successful (2.13.6).

What is your issue?

Xarray raises a FutureWarning in its date_range, also observable in your tests. The precise warning is:

xarray/coding/cftime_offsets.py:1130: FutureWarning: Argument closed is deprecated in favor of inclusive.

You should discuss if you will adapt the new inclusive argument or add a workaround.

• [x] Closes #7312
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

Turns out the default value of arguments is important for overloads, haha.

This PR adds a utils.parse_dims function for parsing one or more dimensions. Currently every function that accepts multiple dimensions does this by itself.

I decided to first see if it would be useful to centralize the dimension parsing and collect inputs before adding it to other functions.

• [x] Potentially closes #7181
• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

I have passed along some more memo dicts, which could prevent some double deep-copying of the same data (don't know how exactly, but who knows :P) Also, I have found some copy calls that did not pass along the deep argument (I am not sure if that breaks things, lets find out). And finally I have found some places where shallow copies are enough.

All together it should improve the performance a lot when copying things around.

This PR adds the mypy option "warn_unused_ignores" which will raise an error if a # type: ignore is used where it is no longer necessary.

Should enable us to keep our types updated.

I am not sure if this will lead to many issues whenever e.g. numpy changes/improves their typing, so we might get errors whenever there is a new version. Maybe it is not that bad, or maybe we can also remove the option again and only do it manually from time to time?

What is your issue?

It seems that some update in urllib3 causes our doctests to fail.

The reason seems to be that botocore uses an interesting construction to import deprecated urllib3 things: python try: # pyopenssl will be removed in urllib3 2.0, we'll fall back to ssl_ at that point. # This can be removed once our urllib3 floor is raised to >= 2.0. with warnings.catch_warnings(): warnings.simplefilter("ignore", category=DeprecationWarning) # Always import the original SSLContext, even if it has been patched from urllib3.contrib.pyopenssl import ( orig_util_SSLContext as SSLContext, ) except ImportError: from urllib3.util.ssl_ import

I assume that this fails because we use -Werror which translates the warning into an error which then is not ignored...

Not sure if this is an issue with botocore or we have to catch this?

• [x] Related to #7270

• [x] Hopefully Closes #6726
• [x] ~Tests added~
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

• [x] Closes #7156
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

• [x] Closes #7275
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~New functions/methods are listed in api.rst~

• [x] Closes #7049
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

• [x] Closes #7158

Technically does not close all fails, but if we close it, the CI will open a new issue anyway and the discussion is not relevant anymore :)

related to https://github.com/pydata/xarray/issues/6726

• silences some user warning from renaming dims.
• fix coverage config

• [x] Closes #6949
• [x] Tests added (typing)
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

What happened?

The plot accessors i.e. dataarray.plot of type _PlotMethods are missing static typing especially of function attributes. See #6947 for an example.

The problem is that many plotting methods are added using hooks via decorators, something that mypy does not understand.

What did you expect to happen?

As a quick fix: type the plot accessors as _PlotMethods | Any to avoid false positives in mypy.

Better to either restructure the accessor with static methods instead of hooks or figure out another way of telling static type checkers about these methods.

Anyway: mypy should not complain.

Minimal Complete Verifiable Example

```Python import xarray as xr

da = xr.DataArray([[1,2,3], [4,5,6]], dims=["x", "y"]) da.plot.contourf(x="x", y="y")

mypy complains:

error: "_PlotMethods" has no attribute "contourf"

```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

On mobile, can edit it later if required. Newest xarray should have this problem, before the accessor was Any.

• [x] Closes #6818
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

Alternative to https://github.com/pydata/xarray/pull/6821

Forgot to add this to #7112

Also, correct the date of the 2022.09 release in whats-new, see #7135

• [x] Closes #7111
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

xarray.testing.assert_identical and probably more do not work yet.

Not sure if that is the correct approach (to simply use a mypy flag) but lets see what people think about it.

While adding type hints to test_backends I noticed that the open_dataset method and the abstract BackendEntrypoint were missing stream types as inputs.

• [x] Closes #2835 (Even though technically it fixes the DataArray version of the issue)
• [x] ~~Tests added ~~ will be done via #7086
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

related to #2839

Mainly to annoy @Illviljan :)

tiny PR that disables coverage on types.py and all typing-only imports a.la. if TYPE_CHECKING: ...

This PR adds the ellipsis typehint to reductions (only where they behave differently from None to reduce overhead). Follow up on https://github.com/pydata/xarray/pull/7017#issuecomment-1243927061

Additionally I was changing a lot of "one or more dimensions" typehints to str | Iterable[Hashable] (See https://github.com/pydata/xarray/issues/6142). Some code changes were necessary to support this fully. Before several things were not working with actual hashable dimensions that are not strings.

• [x] Closes #7078
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

What happened?

We used Data{set,Array}.rename{_vars}({coord: dim_coord}) to make a coordinate a dimension coordinate (instead of set_index). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like sel fail with a strange KeyError.

What did you expect to happen?

I expect one of two things to be true:

1. rename{_vars} does not allow setting dimension coordinates (raises Error and tells you to use set_index)
2. rename{_vars} checks for this occasion and sets the index correctly

Minimal Complete Verifiable Example

```python import xarray as xr

data = xr.DataArray([5, 6, 7], coords={"c": ("x", [1, 2, 3])}, dims="x")

<xarray.DataArray (x: 3)>

array([5, 6, 7])

Coordinates:

c (x) int64 1 2 3

Dimensions without coordinates: x

data_renamed = data.rename({"c": "x"})

<xarray.DataArray (x: 3)>

array([5, 6, 7])

Coordinates:

* x (x) int64 1 2 3

data_renamed.indexes

Empty

data_renamed.sel(x=2)

KeyError: 'no index found for coordinate x'

if we use set_index it works

data_indexed = data.set_index({"x": "c"})

looks the same as data_renamed!

<xarray.DataArray (x: 3)>

array([1, 2, 3])

Coordinates:

* x (x) int64 1 2 3

data_indexed.indexes

x: Int64Index([1, 2, 3], dtype='int64', name='x')

```

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

This PR adds an Ellipsis typehint to some functions.

Interestingly mypy did not complain at the tests before, I assume it is because "..." is Hashable or something like that?

I don't know what to do with reductions, since they also support ellipsis, but it is basically the same as using None. Therefore, I assume it is not necessary to expose this feature.

Did I miss any functions where ellipsis is supported? It is hard to look for "..."... xD

• [x] Closes #6875
• [x] ~~Tests added~~
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

This PR removes several warnings from the tests and improves their typing on the way.

This PR tries to get rid of several warnings in the tests.

I could not get rid of RuntimeWarning: All-NaN slice encountered for tests with dask. Does anyone know why is that? pytest.mark.filterwarnings does not seem to capture them...

• [x] Closes #6931
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

Some typing on the way :)

This PR removes the _SupportsDType dependency from numpy and introduces its own.

Closes https://github.com/pydata/xarray/issues/6818

• [x] Closes #6739
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

I have taken the freedom to move all rolling related tests into their own testing module. https://github.com/pydata/xarray/pull/6730 should then take care of the (by now) copy-pasted da and ds fixtures.

• [x] Closes #5458
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

On the way, I have added the support for changing the name and dims/coords in the same rename call.

Also took the freedom to fix some unrelated typing problems.

What happened: http://xarray.pydata.org/en/stable/generated/xarray.DataArray.rename.html#xarray.DataArray.rename states that:

Returns a new DataArray with renamed coordinates or a new name.

What you expected to happen: It should state: "Returns a new DataArray with renamed coordinates, dimensions or a new name." Since it definitely can do that.

Minimal example xr.DataArray([1, 2, 3]).rename({"dim_0": "new"})

Further While at it: Dataset.rename als does not mention explicitly that you can rename coordinates.

This PR moves all DataArrayRolling and DatasetRolling tests to their own module.

See request https://github.com/pydata/xarray/pull/6744#issuecomment-1182169308

• [x] Closes #6741
• [x] ~~Tests added~~
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] ~~New functions/methods are listed in api.rst~~

This PR adds typing support for groupby, coarsen, rolling, weighted and resample.

There are several open points:

1. Coarsen is missing type annotations for reductions like max, they get added dynamically.
2. The Groupby group-key type is quite wide. Does anyone have any idea on how to type it correctly? For now it is still Any.
3. Several function signatures were inconsistent between the DataArray and Dataset versions (looking at you: map). I took the freedom to align them (required for mypy), hopefully this does not break too much.
4. I was moving the generation functions from DataWithCoords to DataArray and Dataset, which adds some copy-paste of code (I tried to keep it minimal) but was unavoidable for typing support. (Adds the bonus that the corresponding modules are now only imported when required).

• [x] Closes #6714
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Currently returns a Mapping from variable names to dtypes for ALL variables in the Dataset, including coordinates. Possibly better to only return data_vars dtypes? Give me your thoughts on that, it is easy to change.

• [x] Closes #6707
• [x] ~~Tests added~~
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

I don't really know how to test this... Is it ok to leave it untested?

• [x] Closes #6227
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

What happened?

I tried to reduce some dimensions using where (sel did not work in this case) and shorten the dimensions with "drop=True". This works fine on DataArrays and Datasets with only a single dimension but fails as soon as you have a Dataset with two dimensions on different variables. The dimensions are left untouched and you have NaNs in the data, just as if you were using "drop=False" (see example).

I am actually not sure what the expected behavior is, maybe I am wrong and it is correct due to some broadcasting rules?

What did you expect to happen?

I expected that relevant dims are shortened. If the ds.where with "drop=False" all variables along a dimenions have some NaNs, then using "drop=True" I expect these dimensions to be shortened and the NaNs removed.

Minimal Complete Verifiable Example

```python import xarray as xr

this works

ds = xr.Dataset({"a": ("x", [1, 2 ,3])}) ds.where(ds > 2, drop=True)

returns:

<xarray.Dataset>

Dimensions: (x: 1)

Dimensions without coordinates: x

Data variables:

a (x) float64 3.0

this doesn't

ds = xr.Dataset({"a": ("x", [1, 2 ,3]), "b": ("y", [2, 3, 4])}) ds.where(ds > 2, drop=True)

returns:

<xarray.Dataset>

Dimensions: (x: 3, y: 3)

Dimensions without coordinates: x, y

Data variables:

a (x) float64 nan nan 3.0

b (y) float64 nan 3.0 4.0

```

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

• [x] Closes #6623
• [x] Tests added

• [x] Closes #6632

So far we have not found a good way of typing dynamically added custom backends (installed via pip). So the only fallback option is to allow str.

I have decided to leave the Literal as type, since it gives the user a nice list of supported backends in the editor.

• [x] Closes #5945
• [x] Tests added

Feel free to wait with merging after the 2022.05.0.dev0 release :)

• This PR adds typing to all Dataset methods (only the ones in Dataset directly and DataWithCoords).
• On the way it fixes several typing issues and adds small code changes for more consistent typing (hopefully does not break anything).
• All test_dataset tests are now typed.
• As usual there are still some mypy bugs open that prevent all typing changes to be final (especially https://github.com/python/mypy/issues/12846). I choose to remove the TypeVars in these occasions to prevent false positives on the user side.
• Sorry for updating so many files at once, mostly these are the changes of adding from __future__ import annotations and automatic fixes from pyupdate.

If anyone has some typing expertise: I think that the way align is typed is wrong. The current implementation only works for aligning a sequence of objects of the same type, but not mixed DataArrays and Datasets. In some placed I had to add some "#type: ignore"s... I should probably open an issue for that.

• [x] Tests added

This PR improves typing of DataArray class methods. Main change is that T_DataArray is used whenever possible.

I have left everything untouched that would cause problems with mypy (there are some mypy bugs) or require larger typing efforts in other areas.

Main problems are argmin and argmax.

What is your issue?

In the recent typing improvements the engine argument for open_dataset was changed from Str to a Literal of xarrays internal engines. This will cause problems for all third party backend plugins.

We have several possibilities:

1. I don't know if there is a way to know installed backends at type checking time. Then we could add this support. (I doubt this is possible seeing how dynamic these imports are)
2. Is it possible for these plugins to tell type checkers that their engine is valid, i.e. change the type signature of xarrays function? Then we should add a how-to in the docu.
3. Else we should probably revert to using Str.

Any typing experts here that could help?

• [x] Closes #6628
• [x] Tests added

forgot to add defaults to the dtype argument, so the typing defaulted back to the Union definition...

This is initial try to get type hints for str and dt accessors.

I think there is no way of accessing the class at class scope (or is there?), so I had to use plain "DataArray" as the generic type of the accessors. I think that is acceptable for now.

The hack of DatetimeAccessor vs TimedeltaAccessor in the CombinedDatetimelikeAccessor.__new__ is something static typing can handle, so at type-checking time all properties of both accessors are available. If someone has a better idea?

Maybe a common interface class for accessors would be also beneficial?

This PR adds a mypy badge to the README.

Also, nicer alt texts for all other badges.

• [x] Closes #5382

Mypy is not able to compute the overloads of to_netcdf properly (too many Unions). I had to add some # type: ignore

I am not sure about the types of * concat_dim * combine_attrs: the Callable part is still with Anys

(partial) typing for functions full_like, zeros_like, ones_like.

I could not figure out how to properly use TypeVars so many things are "hardcoded" with overloads.

I have added a DTypeLikeSave to npcompat, not sure that this file is supposed to be edited.

Problem1: TypeVar["T", Dataset, DataArray, Variable] can only be one of these three, but never with Union[Dataset, DataArray] which is used in several other places in xarray. Problem2: The official mypy support says: use TypeVar["T", bound=Union[Dataset, DataArray, Variable] but the the isinstance(obj, Dataset) could not be correctly resolved (is that a mypy issue?).

So if anyone can get it to work with TypeVars, feel free to change it. :)

• [x] Closes #6600
• [x] Tests added

• [x] Closes #6597
• [x] Tests added

Attempt to fix the typing issues in xr.polyval.

Some problems are still occuring and require a type: ignore. They seem more like mypy issues to me.

• [x] Closes #6554
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Additionally I have added new literal types for error handling (Only applied to functions related to isel such that mypy stops complaining).

What happened?

When using DataArray/Dataset.isel with drop=True with a scalar DataArray as indexer (see example) resulting scalar coordinates do not get dropped. When using an integer the behavior is as expected.

What did you expect to happen?

I expect that using a scalar DataArray behaves the same as an integer.

Minimal Complete Verifiable Example

```Python import xarray as xr

da = xr.DataArray([1, 2, 3], dims="x", coord={"k": ("x", [0, 1, 2])})

<xarray.DataArray (x: 3)>

array([1, 2, 3])

Coordinates:

k (x) int32 0 1 2

da.isel({"x": 1}, drop=True)

works

<xarray.DataArray ()>

array(2)

da.isel({"x": xr.DataArray(1)}, drop=True)

does not drop "k" coordinate

<xarray.DataArray ()>

array(2)

Coordinates:

k int32 1

```

Relevant log output

No response

Anything else we need to know?

No response

Environment

• [x] Closes #6526, #6411
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

What happened?

I have some polynomial coefficients and want to evaluate them at some values using xr.polyval.

As described in the docstring/docu I created a 1D coordinate DataArray and pass it to xr.polyval but it raises a KeyError (see example).

What did you expect to happen?

I expected that the polynomial would be evaluated at the given points.

Minimal Complete Verifiable Example

```Python import xarray as xr

coeffs = xr.DataArray([1, 2, 3], dims="degree")

With a "handmade" coordinate it fails:

coord = xr.DataArray([0, 1, 2], dims="x")

xr.polyval(coord, coeffs)

raises:

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "xarray/core/computation.py", line 1847, in polyval

x = get_clean_interp_index(coord, coord.name, strict=False)

File "xarray/core/missing.py", line 252, in get_clean_interp_index

index = arr.get_index(dim)

File "xarray/core/common.py", line 404, in get_index

raise KeyError(key)

KeyError: None

If one adds a name to the coord that is called like the dimension:

coord2 = xr.DataArray([0, 1, 2], dims="x", name="x")

xr.polyval(coord2, coeffs)

works

```

Relevant log output

No response

Anything else we need to know?

I assume that the "standard" workflow is to obtain the coord argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow.

It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future.

Environment

Is your feature request related to a problem?

When using sortby with ascending=False on a DataArray/Dataset without an explicit index, the data gets correctly reversed, but it is not possible to tell anymore which ordering the data has.

If an explicit index (like [0, 1, 2]) exists, it gets correctly reordered and allowes correct aligning.

Describe the solution you'd like

For consistency with aligning xarray should create a new index that indicates that the data has been reordered, i.e. [2, 1, 0].

Only downside: this will break code that relies on non-existent indexes.

Describe alternatives you've considered

No response

Additional context

No response

What happened?

When grouping and reducing an array or dataset over a multi-index the coordinates that make up the multi-index get renamed to "{name_of_multiindex}_level_{i}".

It only works correctly when the Multiindex is a "homogenous grid", i.e. as obtained by stacking.

What did you expect to happen?

I expect that all coordinates keep their initial names.

Minimal Complete Verifiable Example

```Python import xarray as xr

this works:

d = xr.DataArray(range(4), dims="t", coords={"x": ("t", [0, 0, 1, 1]), "y": ("t", [0, 1, 0, 1])}) dd = d.set_index({"t": ["x", "y"]})

returns

<xarray.DataArray (t: 4)>

array([0, 1, 2, 3])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1

- y (t) int64 0 1 0 1

dd.groupby("t").mean(...)

returns

<xarray.DataArray (t: 4)>

array([0., 1., 2., 3.])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1

- y (t) int64 0 1 0 1

this does not work

d2 = xr.DataArray(range(6), dims="t", coords={"x": ("t", [0, 0, 1, 1, 0, 1]), "y": ("t", [0, 1, 0, 1, 0, 0])}) dd2 = d2.set_index({"t": ["x", "y"]})

returns

<xarray.DataArray (t: 6)>

array([0, 1, 2, 3, 4, 5])

Coordinates:

* t (t) MultiIndex

- x (t) int64 0 0 1 1 0 1

- y (t) int64 0 1 0 1 0 0

dd2.groupby("t").mean(...)

returns

<xarray.DataArray (t: 4)>

array([2. , 1. , 3.5, 3. ])

Coordinates:

* t (t) MultiIndex

- t_level_0 (t) int64 0 0 1 1

- t_level_1 (t) int64 0 1 0 1

```

Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4

xarray: 0.21.1 pandas: 1.4.0 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.3 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

Is your feature request related to a problem? Please describe. I have a dataset where I do not know which of two dimensions (lets call them a and b) exists in this dataset (So either it has dims ("a", "other") or ("b", "other")). I would like to make sure that this dimension is first using transpose, but currently this is only possible using if or try statements. Just using ds.transpose("a", "b", "other") raises a ValueError arguments to transpose XXX must be permuted dataset dimensions YYY.

Describe the solution you'd like It would be nice if I could just use ds.transpose("a", "b", "other", missing_dims="ignore") similar to how DataArray.transpose handles it.

Describe alternatives you've considered Currently I'm also using ds.map(lambda x: x.transpose("a", "b", "other", missing_dims="ignore")), which could (maybe?) replace the current implementation of the transpose.

While at it, transpose_coords could also be exposed to Dataset.transpose.