A drive-by PR as I was trying to figure out how these functions works.

Enforce typing on all tests in test_plot.py and add the remaining type hints.

Add a new what's new section

Currently using getitem is not allowed for either a arrayfunction or arrayapi according to the type hints. Add support for it.

Issues found in #8344 as more and more typing was added. But getitem is used in main as well which should fail, but manages to pass due to lack of type hints: https://github.com/pydata/xarray/blob/9acc411bc7e99e61269eadf77e96b9ddd40aec9e/xarray/namedarray/core.py#L792

Notes * mypy will now complain for _arrayapi[None] or _arrayapi([None,)], use xp.expand_dims(_arrayapi) for now instead.

https://github.com/numpy/numpy/pull/25022 https://github.com/data-apis/array-api/pull/687

Check if the Array object from numpy.array_api is a valid duckarray. If it isn't we're likely doing something wrong.

Comments: * array_api standard does not specify a __array__-method. * Move __array__ to _arrayfunction only * We therefore have to use np.asarray(namedarray.data) when comparing ndarrays in the tests

Seen in #8344

Add a test for masked array in order to make sure it works. Be consistent and use the shapetype dtype generics style instead.

Starting to think it's easier to just stick to one style of typing, even though the other option might be a little shorter.

Use _DType_co instead, otherwise the dtype wont be correctly passed along. Quick follow up to #8387.

Fixes the following pyright error on main:

C:\Users\J.W\Documents\GitHub\xarray\xarray\namedarray\core.py:846:39 - error: Argument of type "NDArray[Any]" cannot be assigned to parameter "data" of type "duckarray[_ShapeType_co@NamedArray, _DType_co@NamedArray] | Default" in function "_replace"   Type "NDArray[Any]" cannot be assigned to type "duckarray[_ShapeType_co@NamedArray, _DType_co@NamedArray] | Default"     "NDArray[Any]" is incompatible with protocol "_arrayfunction[_ShapeType_co@NamedArray, _DType_co@NamedArray]"       Type "(dtype: None = ..., /) -> ndarray[Any, _dtype]" cannot be assigned to type "(dtype: _DType@__array__, /) -> ndarray[Any, _DType@__array__]"         Parameter 1: type "_DType@__array__" cannot be assigned to type "None"           Type cannot be assigned to type "None"       Type "(dtype: None = ..., /) -> ndarray[Any, _dtype]" cannot be assigned to type "(dtype: _DType@__array__, /) -> ndarray[Any, _DType@__array__]"         Parameter 1: type "_DType@__array__" cannot be assigned to type "None"           Type cannot be assigned to type "None" ... (reportGeneralTypeIssues)

Fixes:

``` Running Sphinx v6.2.1

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8387/lib/python3.10/site-packages/sphinx/config.py", line 353, in eval_config_file code = compile(f.read(), filename.encode(fs_encoding), 'exec') File "/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/8387/doc/conf.py", line 240 navigation_with_keys=False, ^^^^^^^^^^^^^^^^^^^^^^^^^^ SyntaxError: keyword argument repeated: navigation_with_keys

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8387/lib/python3.10/site-packages/sphinx/cmd/build.py", line 280, in build_main app = Sphinx(args.sourcedir, args.confdir, args.outputdir, File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8387/lib/python3.10/site-packages/sphinx/application.py", line 207, in init self.config = Config.read(self.confdir, confoverrides or {}, self.tags) File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8387/lib/python3.10/site-packages/sphinx/config.py", line 177, in read namespace = eval_config_file(filename, tags) File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/8387/lib/python3.10/site-packages/sphinx/config.py", line 357, in eval_config_file raise ConfigError(msg % err) from err sphinx.errors.ConfigError: There is a syntax error in your configuration file: keyword argument repeated: navigation_with_keys (conf.py, line 240)

Configuration error: There is a syntax error in your configuration file: keyword argument repeated: navigation_with_keys (conf.py, line 240) Command time: 0s Return: 2

```

Seen in #8387

Add more chunkedarray typing options. Also clean up some ideas that didn't work out. Using TypeVar for arrays for example has turned out to not work that great because of the very common dtype changes.

If you want to check if an ArrayLike is a chunked array: python if isinstance(data, _chunkedarrayfunction_or_api): data.chunk() However try to rely on mypy in private functions over isinstance checks for better performance: python def _do_compute(data: chunkedduckarray[Any, _DType]) -> duckarray[Any, _DType]: return data.compute()

ndim is usually defined as len(self.shape), which does not care about the values in the shape-tuple. So there's no risk when getting unknown shape sizes from for example a masked dask array.

Reference: https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.ndim.html

Use the normal repr now that NamedArray has one.
Noticed a bug as well, that sometimes the repr attempts to call NamedArray._in_memory which doesn't exist.

Seen in #8344.

Noticed an unnecessary for loop when playing around with self.get_axis_num in #8344.

With more realistic array sizes maybe the improvements in #8339 should be noticeable.

Reverts a few things from #8313. An IndexVariable can only have PandasIndexAdapter as self._data, which makes the for MemoryCachedArray unnecessary.

Seen in #8294.

Seen in #8294.

It's usually better to switch if-checks so that the wider type is defined first. Especially when relying on the implicit type casting like we do here.

This seems to be fixed according to #6551.

Reducing type errors in #8241

A smaller PR related to trying to pass T_DuckArray's along xarray's different classes.

Fixes the mypy error we're seeing in the CI at the moment:

xarray/core/dataset.py:5286: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "Hashable" [type-var]

Can't sort Hashables apparently.

Temporally skip tests in order to make CI pass again, the troublesome tests are reported in #7971.

Following recommendation from: https://github.com/pydata/xarray/pull/7824#discussion_r1196114696

```python

main:

a = tuple(f"dim_{i}" for i in range(500)) %timeit OrderedSet(a) 46 µs ± 2.26 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

This PR:

a = tuple(f"dim_{i}" for i in range(500)) %timeit OrderedSet(a) 28.9 µs ± 476 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each) ```

Split out from #7858.

Improve mypy coverage in order to understand better what the functions are actually used for.

• Use fastpaths
• Only transpose multi-dimensional arrays.
• Calculating chunks and shapes can be costly, prefer doing it once.

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

It's a little tricky getting the mypy version, so adding it to the show_versions function as well.

Removes a warning in the plotting tests, related to https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.5.0.html#auto-removal-of-grids-by-pcolor-and-pcolormesh.

I just added ax.grid(False) but maybe we just want whatever is the default of matplotlib?

```python import xarray as xr

airtemps = xr.tutorial.open_dataset("air_temperature") air = airtemps.air - 273.15 air.attrs = airtemps.air.attrs air.attrs["units"] = "deg C"

air2d = air.isel(time=500) fig, axs = plt.subplots(2,1) air2d.plot(ax=axs[0]) # No grid. air2d.plot(ax=axs[1]) # With grid. axs[1].grid(True) ```

What is your issue?

As I've understood it ZarrBackendEntrypoint is the engine used when loading zarr-files. But for some reason we show ZarrStore in xr.backends. I believe the ZarrStore class is supposed to be just a implementation detail, right?

```python

The available engines:

xr.backends.list_engines() Out[23]: {'netcdf4': <xarray.backends.netCDF4_.NetCDF4BackendEntrypoint object at 0x00000296D05D11F0>, 'h5netcdf': <xarray.backends.h5netcdf_.H5netcdfBackendEntrypoint object at 0x00000296D05D14C0>, 'scipy': <xarray.backends.scipy_.ScipyBackendEntrypoint object at 0x00000296D05D11C0>, 'pseudonetcdf': <xarray.backends.pseudonetcdf_.PseudoNetCDFBackendEntrypoint object at 0x00000296D05D1430>, 'pydap': <xarray.backends.pydap_.PydapBackendEntrypoint object at 0x00000296D05D1310>, 'store': <xarray.backends.store.StoreBackendEntrypoint object at 0x00000296D05D1340>, 'zarr': <xarray.backends.zarr.ZarrBackendEntrypoint object at 0x00000296D05D12E0>}

The public class is ZarrStore instead of ZarrBackendEntrypoint, how come?

dir(xr.backends) Out[22]: ['AbstractDataStore', 'BackendArray', 'BackendEntrypoint', 'CachingFileManager', 'CfGribDataStore', 'DummyFileManager', 'FileManager', 'H5NetCDFStore', 'InMemoryDataStore', 'NetCDF4DataStore', 'NioDataStore', 'PseudoNetCDFDataStore', 'PydapDataStore', 'ScipyDataStore', 'ZarrStore', 'all', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'path', 'spec', 'api', 'cfgrib_', 'common', 'file_manager', 'h5netcdf_', 'list_engines', 'locks', 'lru_cache', 'memory', 'netCDF4_', 'netcdf3', 'plugins', 'pseudonetcdf_', 'pydap_', 'pynio_', 'rasterio_', 'scipy_', 'store', 'zarr'] ```

This test has been quite troublesome and since .open_rasterio has been deprecated since 0.20 I think it's fine removing the test.

• Fix outdated typing.
• Clearer variable names.

These changes should hopefully yield some performance improvements.

Prepare to be able to use a .set_zlabels in facetgrids. Reduces diffs in #6778.

Minor tweak to reduce diffs in #6778.

Some minor import oddities I noticed while reading the code.

Use math instead of numpy where possible, looks better and seems faster as well. Idea from https://github.com/pydata/xarray/pull/6702#discussion_r903986442, https://github.com/dask/dask/pull/9232

Example: ```python %timeit math.prod((1, 2, 3)) 161 ns ± 10.5 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

%timeit int(np.prod((1, 2, 3))) 11.9 µs ± 248 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) ```

The shape methods can have multiple integers inside the tuple, this fixes that and adds to typing to all shape methods I found.

Back in 2017 the register_pandas_datetime_converter_if_needed was added to workaround matplotlib crashing when handling datetime objects, (#1669). The example that crashed back then was (#1661): ```python import xarray import numpy

da = xarray.DataArray( numpy.arange(3*4).reshape(3,4), dims=("x", "y"), coords={"x": [1,2,3], "y": [numpy.datetime64(f"2000-01-{x:02d}") for x in range(1, 5)]})

da.plot.pcolormesh() ``` This has probably been working by default in matplotlib since 2.2.0, when datetime support was added https://github.com/matplotlib/matplotlib/releases/tag/v2.2.0. (https://github.com/matplotlib/matplotlib/pull/9779)

Related issues: https://github.com/mwaskom/seaborn/issues/1325

• [x] Closes #6102
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Reduce diffs in #6778.

Reduce diffs in #6778. _interval_to_double_bound_points was returning a list instead of ndarray which I had some issues with.

Reduce diffs in #6778. I remember finding it annoying that I got a list of the valid strings but not the actual string used.

Fixes: xarray/tests/test_backends_file_manager.py:56: error: Incompatible types in assignment (expression has type "_GeneratorContextManager[Any]", variable has type "WarningsChecker") [assignment] Just added a ignore.

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

What happened: Dataset.interp silently drops boolean variables.

What you expected to happen: If I'm interpolating a group of variables I expect to get all of them back in the correct shape with relevant values in them.

If the variables are boolean or object arrays I don't expect it to do linear interpolation because it doesn't make sense but stepwise interpolation like nearest or zero order interpolation should be fine to expect.

Minimal Complete Verifiable Example:

```python import numpy as np a = np.arange(0, 5) b = np.core.defchararray.add("long_variable_name", a.astype(str)) coords = dict(time=da.array([0, 1])) data_vars = dict() for v in b: data_vars[v] = xr.DataArray( name=v, data=np.array([0, 1]).astype(bool), dims=["time"], coords=coords, ) ds1 = xr.Dataset(data_vars)

Print raw data:

print(ds1) Out[3]: <xarray.Dataset> Dimensions: (time: 2) Coordinates: * time (time) int32 0 1 Data variables: long_variable_name0 (time) bool False True long_variable_name1 (time) bool False True long_variable_name2 (time) bool False True long_variable_name3 (time) bool False True long_variable_name4 (time) bool False True

Interpolate:

ds1 = ds1.interp( time=da.array([0, 0.5, 1, 2]), assume_sorted=True, method="nearest", kwargs=dict(fill_value="extrapolate"), )

Print interpolated data:

<xarray.Dataset> Dimensions: (time: 4) Coordinates: * time (time) float64 0.0 0.5 1.0 2.0 Data variables: empty ```

Anything else we need to know?: ds.interp_likeuse ds.reindex in these cases which seems like a good choice in ds.interp as well. But I think that both ds.interp and ds.interp_like should fill by default with nearest value instead of np.nan because we're still requesting interpolation.

Environment:

Is your feature request related to a problem? Please describe. I've been working on opening custom hdf formats in xarray, reading up on the apiv2 it is currently only possible to declare a new external plugin in setup.py but that doesn't seem easy or intuitive to me.

Describe the solution you'd like Why can't we simply be allowed to add functions to the engine parameter? Example: ```python from custom_backend import engine

ds = xr.load_dataset(filename, engine=engine) ``` This seems like a small function change to me from my initial quick look because there's mainly a bunch of string checks in the normal case until we get to the registered backend functions, if we send in a function instead in the engine-parameter we can just bypass those checks.