• [ ] Towards #8856
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

this PR builds towards

• https://github.com/pydata/xarray/pull/8870
• https://github.com/pydata/xarray/pull/8856

the primary objective is to partially address

1. Implement fall back .oindex, .vindex properties on BackendArray base class. These will simply rewrap the key tuple with the appropriate *Indexer object, and pass it on to __getitem__ or __setitem__. These methods will also raise DeprecationWarning so that external backends will know to migrate to .oindex, and .vindex over the next year.

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [ ] Towards #8856
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

this is another follow-up to previous Prs

• 8780

• 8750

• 8790

, continuing the proposal outlined in plan for decoupling lazy indexing functionality from NamedArray

Cc @maxrjones for visibility

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

this is a follow-up to previous PRs (#8780 and #8750), continuing the efforts outlined in the plan for decoupling lazy indexing functionality from NamedArray. the primary focus of this PR is the removal of vectorized and orthogonal indexing logic from the __getitem__ method. Now, __getitem__ exclusively handles basic indexers, aligning with points 3 and 4 of the plan:

• as per point 3, lazy indexing arrays will now implement __getitem__ solely for basic indexing. For orthogonal indexing, .oindex will be used, and for vectorized indexing, .vindex will be utilized.
• following point 4, IndexingAdapter classes have been updated to consistently implement __getitem__, .oindex, and .vindex

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• 8827 follow up to fix https://readthedocs.org/projects/xray/builds/23743522/

What is your issue?

Recent changes in Dask's tokenization process (https://github.com/dask/dask/pull/10876) seem to have introduced unexpected behavior in Xarray's test suite. This has led to CI failures, specifically in tests related to tokenization.

• https://github.com/pydata/xarray/actions/runs/8069874717/job/22045898877

```python ---------- coverage: platform linux, python 3.12.2-final-0 ----------- Coverage XML written to file coverage.xml

=========================== short test summary info ============================ FAILED xarray/tests/test_dask.py::test_token_identical[obj0-<lambda>1] - AssertionError: assert 'bbd9679bdaf2...d3db65e29a72d' == '6352792990cf...e8004a9055314'

• 6352792990cfe23adb7e8004a9055314

• bbd9679bdaf284c371cd3db65e29a72d FAILED xarray/tests/test_dask.py::test_token_identical[obj0-<lambda>2] - AssertionError: assert 'bbd9679bdaf2...d3db65e29a72d' == '6352792990cf...e8004a9055314'

• 6352792990cfe23adb7e8004a9055314

• bbd9679bdaf284c371cd3db65e29a72d FAILED xarray/tests/test_dask.py::test_token_identical[obj1-<lambda>1] - AssertionError: assert 'c520b8516da8...0e9e0d02b79d0' == '9e2ab1c44990...6ac737226fa02'

• 9e2ab1c44990adb4fb76ac737226fa02

• c520b8516da8b6a98c10e9e0d02b79d0 FAILED xarray/tests/test_dask.py::test_token_identical[obj1-<lambda>2] - AssertionError: assert 'c520b8516da8...0e9e0d02b79d0' == '9e2ab1c44990...6ac737226fa02'

• 9e2ab1c44990adb4fb76ac737226fa02

• c520b8516da8b6a98c10e9e0d02b79d0 = 4 failed, 16293 passed, 628 skipped, 90 xfailed, 71 xpassed, 213 warnings in 472.07s (0:07:52) = Error: Process completed with exit code 1. ```

previously, the following code snippet would pass, verifying the consistency of tokenization in Xarray objects:

```python In [1]: import xarray as xr, numpy as np

In [2]: def make_da(): ...: da = xr.DataArray( ...: np.ones((10, 20)), ...: dims=["x", "y"], ...: coords={"x": np.arange(10), "y": np.arange(100, 120)}, ...: name="a", ...: ).chunk({"x": 4, "y": 5}) ...: da.x.attrs["long_name"] = "x" ...: da.attrs["test"] = "test" ...: da.coords["c2"] = 0.5 ...: da.coords["ndcoord"] = da.x * 2 ...: da.coords["cxy"] = (da.x * da.y).chunk({"x": 4, "y": 5}) ...: ...: return da ...:

In [3]: da = make_da()

In [4]: import dask.base

In [5]: assert dask.base.tokenize(da) == dask.base.tokenize(da.copy(deep=False))

In [6]: assert dask.base.tokenize(da) == dask.base.tokenize(da.copy(deep=True))

In [9]: dask.version Out[9]: '2023.3.0' ```

However, post-update in Dask version '2024.2.1', the same code fails:

```python In [55]: ...: def make_da(): ...: da = xr.DataArray( ...: np.ones((10, 20)), ...: dims=["x", "y"], ...: coords={"x": np.arange(10), "y": np.arange(100, 120)}, ...: name="a", ...: ).chunk({"x": 4, "y": 5}) ...: da.x.attrs["long_name"] = "x" ...: da.attrs["test"] = "test" ...: da.coords["c2"] = 0.5 ...: da.coords["ndcoord"] = da.x * 2 ...: da.coords["cxy"] = (da.x * da.y).chunk({"x": 4, "y": 5}) ...: ...: return da ...:

In [56]: da = make_da() ```

```python In [57]: assert dask.base.tokenize(da) == dask.base.tokenize(da.copy(deep=False))

AssertionError Traceback (most recent call last) Cell In[57], line 1 ----> 1 assert dask.base.tokenize(da) == dask.base.tokenize(da.copy(deep=False))

AssertionError:

In [58]: dask.base.tokenize(da) Out[58]: 'bbd9679bdaf284c371cd3db65e29a72d'

In [59]: dask.base.tokenize(da.copy(deep=False)) Out[59]: '6352792990cfe23adb7e8004a9055314'

In [61]: dask.version Out[61]: '2024.2.1' ```

additionally, a deeper dive into dask.base.normalize_token() across the two Dask versions revealed that the latest version includes additional state or metadata in tokenization that was not present in earlier versions.

• old version python In [29]: dask.base.normalize_token((type(da), da._variable, da._coords, da._name)) Out[29]: ('tuple', [xarray.core.dataarray.DataArray, ('tuple', [xarray.core.variable.Variable, ('tuple', ['x', 'y']), 'xarray-<this-array>-14cc91345e4b75c769b9032d473f6f6e', ('list', [('tuple', ['test', 'test'])])]), ('list', [('tuple', ['c2', ('tuple', [xarray.core.variable.Variable, ('tuple', []), (0.5, dtype('float64')), ('list', [])])]), ('tuple', ['cxy', ('tuple', [xarray.core.variable.Variable, ('tuple', ['x', 'y']), 'xarray-<this-array>-8e98950eca22c69d304f0a48bc6c2df9', ('list', [])])]), ('tuple', ['ndcoord', ('tuple', [xarray.core.variable.Variable, ('tuple', ['x']), 'xarray-ndcoord-82411ea5e080aa9b9f554554befc2f39', ('list', [])])]), ('tuple', ['x', ('tuple', [xarray.core.variable.IndexVariable, ('tuple', ['x']), ['x', ('603944b9792513fa0c686bb494a66d96c667f879', dtype('int64'), (10,), (8,))], ('list', [('tuple', ['long_name', 'x'])])])]), ('tuple', ['y', ('tuple', [xarray.core.variable.IndexVariable, ('tuple', ['y']), ['y', ('fc411db876ae0f4734dac8b64152d5c6526a537a', dtype('int64'), (20,), (8,))], ('list', [])])])]), 'a'])

• most recent version

python In [44]: dask.base.normalize_token((type(da), da._variable, da._coords, da._name)) Out[44]: ('tuple', [('7b61e7593a274e48', []), ('tuple', [('215b115b265c420c', []), ('tuple', ['x', 'y']), 'xarray-<this-array>-980383b18aab94069bdb02e9e0956184', ('dict', [('tuple', ['test', 'test'])])]), ('dict', [('tuple', ['c2', ('tuple', [('__seen', 2), ('tuple', []), ('6825817183edbca7', ['48cb5e118059da42']), ('dict', [])])]), ('tuple', ['cxy', ('tuple', [('__seen', 2), ('tuple', ['x', 'y']), 'xarray-<this-array>-6babb4e95665a53f34a3e337129d54b5', ('dict', [])])]), ('tuple', ['ndcoord', ('tuple', [('__seen', 2), ('tuple', ['x']), 'xarray-ndcoord-8636fac37e5e6f4401eab2aef399f402', ('dict', [])])]), ('tuple', ['x', ('tuple', [('abc1995cae8530ae', []), ('tuple', ['x']), ['x', ('99b2df4006e7d28a', ['04673d65c892b5ba'])], ('dict', [('tuple', ['long_name', 'x'])])])]), ('tuple', ['y', ('tuple', [('__seen', 25), ('tuple', ['y']), ['y', ('88974ea603e15c49', ['a6c0f2053e85c87e'])], ('dict', [])])])]), 'a'])

Cc @dcherian / @crusaderky for visibility

• [ ] #8750 follow up
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

this pull request represents an initial effort to refactor the indexing logic within indexing.py by separating it from variable.py. to enhance the flexibility of explicitly indexed array classes, this PR introduces orthogonal ~~and vectorized~~ indexing methods—.oindex[key] ~~and .vindex[key[~~. these changes are a part of a broader initiative aimed at enabling indexing support for namedarray

• [x] Towards https://github.com/pydata/xarray/issues/8238
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

edit 1: - to minimize the blast radius of this PR, i decided to just focus on oindex[key] method in this PR. i intend to introduce vindex[key] in a separate PR

TODO:

• [x] https://github.com/pydata/xarray/pull/8750#discussion_r1491169460
• [ ] Enable passing the underlying tuple key

this pull request removes the "unnecessary optimization" introduced in https://github.com/pydata/xarray/pull/3588 that attempts to rewrite the indexer key for BasicIndexer and OuterIndexer

• [x] Closes #8377
• [ ] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

This PR contains the design document for "named-array" a proposed new package aimed at providing a lightweight, efficient array structure with named dimensions. the main objective of named-array is to provide a more accessible version of Xarray's Variable class without the heavy dependencies like Pandas.

~~## proposed name~~

~~The "Xarray-lite" name is a placeholder for the time being. we have considered two main candidates for the package's name: "narray" and "yarray." feedback on the proposed names or other suggestions is welcome.~~

Related issues:

• https://github.com/pydata/xarray/issues/5081
• https://github.com/pydata/xarray/issues/3981

this PR copies the dtypes module from the xarray.core package to the xarray.namedarray package. It also recreates the ReprObject class from the utils module within the xarray.namedarray package

These changes are part of the ongoing work in: - https://github.com/pydata/xarray/pull/8244

• [ ] towards #8238
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• ref: https://github.com/xarray-contrib/xarray.dev/issues/66#issuecomment-1664561089

Cc @jhamman

really like how good these look on both light/dark backgrounds:

What is your issue?

A recent release of pytest (v8.0.0) appears to have broken our CI.

bash pytest 8.0.0 pyhd8ed1ab_0 conda-forge pytest-cov 4.1.0 pyhd8ed1ab_0 conda-forge pytest-env 1.1.3 pyhd8ed1ab_0 conda-forge pytest-github-actions-annotate-failures 0.2.0 pypi_0 pypi pytest-timeout 2.2.0 pyhd8ed1ab_0 conda-forge pytest-xdist 3.5.0 pyhd8ed1ab_0 conda-forge

Strangely, the issue doesn't seem to occur when using previous versions (e.g. v7.4.4). our last successful CI run used pytest v7.4.4

bash pytest 7.4.4 pyhd8ed1ab_0 conda-forge pytest-cov 4.1.0 pyhd8ed1ab_0 conda-forge pytest-env 1.1.3 pyhd8ed1ab_0 conda-forge pytest-github-actions-annotate-failures 0.2.0 pypi_0 pypi pytest-timeout 2.2.0 pyhd8ed1ab_0 conda-forge pytest-xdist 3.5.0 pyhd8ed1ab_0 conda-forge

i recreated the environment and successfully ran tests locally. the CI failures appear to be connected to the latest release of pytest. i haven't had a chance to do an in-depth exploration of the changes from pytest which could be influencing this disruption. so, i wanted to open an issue to track what is going on. in the meantime, i'm going to pin pytest to an earlier version.

any insights, especially from those familiar with changes in the pytest v8.0.0 update, are warmly welcomed.

• [ ] towards fixing #8681
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

this is the initial phase of our plan to split the .set_dims() method into two distinct methods .expand_dims() and .broadcast_to() as outlined in the design document. - [x] add expand_dims() - [x] add broadcast_to() - [x] add .transpose() and .T to NamedArray - [x] update whats-new.rst

• [x] Closes #2620
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [ ] #8365 follow up
• [ ] Tests added

this pull request introduces the initial prototype for NamedArray as discussed in issue #8073. the intention is to approach the implementation in multiple phases through separate pull requests. the main objectives of this PR are as follows:

1. creation of a new class, xarray.named_array.NamedArray, which will encompass most of the necessary functionality for the lightweight version of xarray.Variable. As a result, xarray.Variable will inherit from this class and implement specific functionality related to xarray.

2. removal of all imports of internal Xarray classes and utilities from the xarray.named_array sub-package. the aim is to eliminate dependencies on anything within xarray.core. This will facilitate the future goal of moving named_array into its own standalone package, with xarray then depending on it.

3. [ ] Towards #3981

4. [x] Tests added
5. [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
6. [ ] New functions/methods are listed in api.rst

• [x] Closes #6649
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

This PR introduces small changes to enhance code maintainability...

• [x] Passes pre-commit run --all-files

• [ ] Addresses https://github.com/pydata/xarray/pull/5139#issuecomment-817362317
• [x] Passes pre-commit run --all-files

Here's a snippet of how this looks like when a pull request is created on the same main repository:

Cc @keewis

Libraries, including xarray, register backends via entrypoints. Xarray adds an EntryPoint "group" in its setup.py. Third party libraries add items to that group in their setup.pys.

Xarray and the third-party library don't need to talk to each other directly to register a backend. They talk through pkg_resources.

For instance, hvplot/holoviews would add an EntryPoint for xarray_plotting_backends in its setup.py:

python entry_points={"xarray_plotting_backends": ['holoviews = hvplot:plotting']}

and xarray users should be able to achieve what @jsignell proposed in #3553:

```python import xarray as xr air = xr.tutorial.open_dataset('air_temperature').load().air xr.set_options(plotting_backend='holoviews')

air.isel(time=500).plot() ```

• [ ] Closes #3553
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #7299
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

This PR removes the existing team page from the docs and links to the newly created https://xarray.dev/team on the landing page

• [ ] Addresses https://github.com/pydata/xarray/pull/6884#issuecomment-1207094758
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [ ] #6530 follow-up

This PR supersedes https://github.com/pydata/xarray/pull/4540. This is still a very rough draft :)

• Closes
• [x] Closes #4473
• [x] Closes #4498
• [x] Closes #659
• [x] Closes #2237
• [x] xref https://github.com/pangeo-data/pangeo/issues/271
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

TODO: - [x] flox vs legacy tests with nans - [x] test passing flox-specific kwargs like engine and method.

xr.Dataset reduce methods such as mean drop non-numeric data variables prior to the reduction. However, as far as I can tell this info isn't mentioned anywhere in the documentation/docstrings. I think this would be useful information to include here for example:

```python In [47]: import xarray as xr

In [48]: import numpy as np, pandas as pd

In [50]: ds['foo'] = xr.DataArray(np.arange(6).reshape(2, 3), dims=['x', 'y'])

In [53]: ds['bar'] = xr.DataArray(pd.date_range(start='2000', periods=6).values.reshape(2, 3), dims=['x', 'y'])

In [54]: ds Out[54]: <xarray.Dataset> Dimensions: (x: 2, y: 3) Dimensions without coordinates: x, y Data variables: foo (x, y) int64 0 1 2 3 4 5 bar (x, y) datetime64[ns] 2000-01-01 2000-01-02 ... 2000-01-06 ```

```python In [55]: ds.mean('x') Out[55]: <xarray.Dataset> Dimensions: (y: 3) Dimensions without coordinates: y Data variables: foo (y) float64 1.5 2.5 3.5

In [56]: ds.bar.mean('x') Out[56]: <xarray.DataArray 'bar' (y: 3)> array(['2000-01-02T12:00:00.000000000', '2000-01-03T12:00:00.000000000', '2000-01-04T12:00:00.000000000'], dtype='datetime64[ns]') Dimensions without coordinates: y ```

• [x] Addresses https://github.com/pydata/xarray/issues/6027#issuecomment-1048737617

Cc @benbovy & @max-sixty

This PR ensures all CI workflows are only enabled on the main repository. A few workflows were still being triggered on forks.

• [x] Closes #6222
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [ ] Closes #6027

Preview:

We've been using pre-commit for the last month or so, and it seems to be working fine. This PR removes the redundant pre-commit hooks GHA workflow.

• [ ] Towards https://github.com/pydata/xarray/issues/5940

This fixes https://github.com/pydata/xarray/issues/5232#issuecomment-833729171

Currently, the CI runs both on a user's fork and pydata/xarray. This leads to duplicate runs and the user is flooded with GitHub notifications in case of failures... This PR addresses this issue by making sure the workflows are triggered on pydata/xarray only.

It turns out that pushes on main branch result in an invalid package version. To address this issue, I am disabling the push event trigger on the release workflow

Cc @keewis

```html Uploading xarray-0.17.1.dev108+gf5e4fd5f-py3-none-any.whl

0%| | 0.00/796k [00:00<?, ?B/s] 73%|███████▎ | 584k/796k [00:00<00:00, 3.39MB/s] 100%|██████████| 796k/796k [00:00<00:00, 1.52MB/s] Content received from server:

400 '0.17.1.dev108+gf5e4fd5f' is an invalid value for Version. Error: Can't use PEP 440 local versions. See packaging.python.org/specifications/core-metadata for more information.

HTTPError: 400 Bad Request from test.pypi.org/legacy '0.17.1.dev108+gf5e4fd5f' is an invalid value for Version. Error: Can't use PEP 440 local versions. See packaging.python.org/specifications/core-metadata for more information. ```

This addresses https://github.com/pydata/xarray/issues/5232#issuecomment-830242266, and will (hopefully) reduce the time it takes to cut a new release.

• [x] Passes pre-commit run --all-files

Follow up PR for #5174

• [x] Tests added
• [x] Passes pre-commit run --all-files

• [x] Closes #5248
• [x] Passes pre-commit run --all-files

As I understand it, the issue described in #5248 is caused by bad indentation... With this PR, we get the following:

@max-sixty, let me know if there are other instances that need to be fixed as part of this PR....

• [x] Passes pre-commit run --all-files

• This PR reduces the max-width for .container divs so as to address issues with RTD's multiple version badges.

• Latest: https://xarray.pydata.org/en/latest/index.html: the rtd badge box doesn't work

• This PR: https://xray--5039.org.readthedocs.build/en/5039/index.html: appears to be working fine

• Fixes CSS properties to enable appropriate toggle/responsiveness when using small screens. Unfortunately, this doesn't seem to have fixed the problem on small screens:

• [x] Passes pre-commit run --all-files

• [ ] Closes #xxxx
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

The styfle/cancel-workflow-action requires a token with write permissions in order to cancel duplicate runs. It turns out that the default GitHub token access made available to pull requests from forks has read permissions only. To address this issue, this PR introduces a new workflow that uses a special workflow_run trigger which allows the workflow to run to run with a token with write permissions.

• [x] Passes pre-commit run --all-files

Cc @keewis

As discussed in today's dev meeting, it'd be convenient to migrate all our CIs from azure pipelines to GitHub Actions.

• [x] Consolidate all style/type checks (isort, black, mypy, flake8) into a single workflow and use the pre-commit GitHub action.
• [x] Create a main CI workflow and parametrize it for different settings
• [x] Add minimum version policy workflow
• [x] Add Doctests workflow
• [x] Remove azure pipelines config
• [x] Update docs: remove references to azure pipelines

As discussed in today's dev meeting, this PR modifies the push and pull_request event triggers by making sure the upstream dev CI is triggered by cron schedule by default.

• [ ] Passes isort . && black . && mypy . && flake8
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [x] Closes #2840
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [x] Closes #4603
• [x] Closes #4643
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Currently, xarray has a room on Gitter: https://gitter.im/pydata/xarray. This room works fine for discussions outside of the codebase. However, Gitter has a few disadvantages:

• The contents are not indexed by search engines
• Searching through existing discussions is almost impossible
• Linking to prior conversations in the room is also complicated

A few months ago, GitHub announced GitHub discussions which is meant to serve as a forum for discussions outside of the codebase. I am of the opinion that GitHub discussions is a better alternative to Gitter. I am wondering if xarray folks would be interested in enabling GitHub discussion on this repo, and migrating general discussions from Gitter to GitHub discussions?

GitHub Discussions is still in beta, but projects can request early access here

Here is a list of a few projects with beta access:

• https://github.com/vercel/vercel/discussions
• https://github.com/KaTeX/KaTeX/discussions
• https://github.com/vercel/next.js/discussions

Towards https://github.com/pydata/xarray/issues/4549

• [ ] Closes #xxxx
• [ ] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

This GitHub action will run tests against the upstream dependencies. When the tests fail, the GH action will create an issue (with "CI" label). The body of the created issue includes:

• test summary report
• a link to the failed workflow run

Here's a couple screenshots from a toy repository:

• [x] Closes #4574
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

What happened:

While computing averaged time using time_bounds via times = bounds.mean('d2'), I get weird results (see example below). It's my understanding that this is a bug, but I don't know yet where it's coming from. I should note that in addition to getting wrong time values, the resulting time values are not monotonically increasing even though my time bounds are.

What you expected to happen:

Correct averaged time values

Minimal Complete Verifiable Example:

```python In [1]: import xarray as xr

In [2]: import numpy as np

In [3]: dates = xr.cftime_range(start='0400-01', end='2101-01', freq='120Y', calendar='noleap')

In [4]: bounds = xr.DataArray(np.vstack([dates[:-1], dates[1:]]).T, dims=['time', 'd2'])

In [5]: bounds
Out[5]: <xarray.DataArray (time: 14, d2: 2)> array([[cftime.DatetimeNoLeap(400, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(520, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(520, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(640, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(640, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(760, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(760, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(880, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(880, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1000, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1000, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1120, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1120, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1240, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1240, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1360, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1360, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1480, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1480, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1600, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1600, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1720, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1720, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1840, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1840, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(1960, 12, 31, 0, 0, 0, 0)], [cftime.DatetimeNoLeap(1960, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(2080, 12, 31, 0, 0, 0, 0)]], dtype=object) Dimensions without coordinates: time, d2

In [6]: bounds.mean('d2')
Out[6]: <xarray.DataArray (time: 14)> array([cftime.DatetimeNoLeap(460, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(580, 12, 31, 0, 0, 0, 0), cftime.DatetimeNoLeap(116, 1, 21, 0, 25, 26, 290448), cftime.DatetimeNoLeap(236, 1, 21, 0, 25, 26, 290448), cftime.DatetimeNoLeap(356, 1, 21, 0, 25, 26, 290448), cftime.DatetimeNoLeap(476, 1, 21, 0, 25, 26, 290448), cftime.DatetimeNoLeap(596, 1, 21, 0, 25, 26, 290448), cftime.DatetimeNoLeap(131, 2, 11, 0, 50, 52, 580897), cftime.DatetimeNoLeap(251, 2, 11, 0, 50, 52, 580897), cftime.DatetimeNoLeap(371, 2, 11, 0, 50, 52, 580897), cftime.DatetimeNoLeap(491, 2, 11, 0, 50, 52, 580897), cftime.DatetimeNoLeap(611, 2, 11, 0, 50, 52, 580897), cftime.DatetimeNoLeap(146, 3, 4, 1, 16, 18, 871345), cftime.DatetimeNoLeap(266, 3, 4, 1, 16, 18, 871345)], dtype=object) Dimensions without coordinates: time

```

Anything else we need to know?:

Environment:

• [x] Closes #3609
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

As pointed out in https://github.com/jbusecke/cmip6_preprocessing/issues/5, xarray appears to be very strict during the encoding and decoding steps even when there are (harmless) discrepancies between missing_value and _FillValue. For instance, when dtypes of missing_value and _FillValue are different, xarray gives up:

```python In [74]: from xarray.coding import variables

In [75]: import numpy as np

In [76]: import xarray as xr

In [77]: original = xr.Variable( ...: ("x",), ...: [0.0, -1.0, 1.0], ...: encoding={"_FillValue": np.float32(1e20), "missing_value": np.float64(1e20)}, ...: )

In [78]: coder = variables.CFMaskCoder()

In [79]: encoded = coder.encode(original)

ValueError Traceback (most recent call last) <ipython-input-79-9fbc3632e28b> in <module> ----> 1 encoded = coder.encode(original)

/glade/work/abanihi/devel/pangeo/xarray/xarray/coding/variables.py in encode(self, variable, name) 156 raise ValueError( 157 "Variable {!r} has multiple fill values {}. " --> 158 "Cannot encode data. ".format(name, [fv, mv]) 159 ) 160

ValueError: Variable None has multiple fill values [1e+20, 1e+20]. Cannot encode data. ```

• [ ] Closes #xxxx
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Cc @dcherian

• Don't need dayofwk=-1 for cftime>=1.0.4

• [x] Closes #3426 and Fixes #3427
• [ ] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I just updated to the latest cftime release, and all of a sudden sel() appears to be broken:

```python In [1]: import xarray as xr

In [2]: import cftime

In [3]: ds = xr.tutorial.load_dataset('rasm')

In [4]: ds
Out[4]: <xarray.Dataset> Dimensions: (time: 36, x: 275, y: 205) Coordinates: * time (time) object 1980-09-16 12:00:00 ... 1983-08-17 00:00:00 xc (y, x) float64 189.2 189.4 189.6 189.7 ... 17.65 17.4 17.15 16.91 yc (y, x) float64 16.53 16.78 17.02 17.27 ... 28.26 28.01 27.76 27.51 Dimensions without coordinates: x, y Data variables: Tair (time, y, x) float64 nan nan nan nan nan ... 29.8 28.66 28.19 28.21 Attributes: title: /workspace/jhamman/processed/R1002RBRxaaa01a/l... institution: U.W. source: RACM R1002RBRxaaa01a output_frequency: daily output_mode: averaged convention: CF-1.4 references: Based on the initial model of Liang et al., 19... comment: Output from the Variable Infiltration Capacity... nco_openmp_thread_number: 1 NCO: "4.6.0" history: Tue Dec 27 14:15:22 2016: ncatted -a dimension...

In [5]: ds.sel(time=slice("1980", "1982"))

ValueError Traceback (most recent call last) <ipython-input-5-2c26e36a673a> in <module> ----> 1 ds.sel(time=slice("1980", "1982"))

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/core/dataset.py in sel(self, indexers, method, tolerance, drop, **indexers_kwargs) 1998 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "sel") 1999 pos_indexers, new_indexes = remap_label_indexers( -> 2000 self, indexers=indexers, method=method, tolerance=tolerance 2001 ) 2002 result = self.isel(indexers=pos_indexers, drop=drop)

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/core/coordinates.py in remap_label_indexers(obj, indexers, method, tolerance, **indexers_kwargs) 390 391 pos_indexers, new_indexes = indexing.remap_label_indexers( --> 392 obj, v_indexers, method=method, tolerance=tolerance 393 ) 394 # attach indexer's coordinate to pos_indexers

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/core/indexing.py in remap_label_indexers(data_obj, indexers, method, tolerance) 259 coords_dtype = data_obj.coords[dim].dtype 260 label = maybe_cast_to_coords_dtype(label, coords_dtype) --> 261 idxr, new_idx = convert_label_indexer(index, label, dim, method, tolerance) 262 pos_indexers[dim] = idxr 263 if new_idx is not None:

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/core/indexing.py in convert_label_indexer(index, label, index_name, method, tolerance) 123 _sanitize_slice_element(label.start), 124 _sanitize_slice_element(label.stop), --> 125 _sanitize_slice_element(label.step), 126 ) 127 if not isinstance(indexer, slice):

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/pandas/core/indexes/base.py in slice_indexer(self, start, end, step, kind) 5032 slice(1, 3) 5033 """ -> 5034 start_slice, end_slice = self.slice_locs(start, end, step=step, kind=kind) 5035 5036 # return a slice

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/pandas/core/indexes/base.py in slice_locs(self, start, end, step, kind) 5246 start_slice = None 5247 if start is not None: -> 5248 start_slice = self.get_slice_bound(start, "left", kind) 5249 if start_slice is None: 5250 start_slice = 0

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind) 5158 # For datetime indices label may be a string that has to be converted 5159 # to datetime boundary according to its resolution. -> 5160 label = self._maybe_cast_slice_bound(label, side, kind) 5161 5162 # we need to look up the label

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/coding/cftimeindex.py in _maybe_cast_slice_bound(self, label, side, kind) 336 pandas.tseries.index.DatetimeIndex._maybe_cast_slice_bound""" 337 if isinstance(label, str): --> 338 parsed, resolution = _parse_iso8601_with_reso(self.date_type, label) 339 start, end = _parsed_string_to_bounds(self.date_type, resolution, parsed) 340 if self.is_monotonic_decreasing and len(self) > 1:

~/opt/miniconda3/envs/intake-esm-dev/lib/python3.7/site-packages/xarray/coding/cftimeindex.py in _parse_iso8601_with_reso(date_type, timestr) 114 # 1.0.3.4. 115 replace["dayofwk"] = -1 --> 116 return default.replace(**replace), resolution 117 118

cftime/_cftime.pyx in cftime._cftime.datetime.replace()

ValueError: Replacing the dayofyr or dayofwk of a datetime is not supported.

```

Output of xr.show_versions()

Expected Output

I can confirm that everything works just fine with an older version of cftime:

```python In [4]: ds.sel(time=slice("1980", "1982"))
Out[4]: <xarray.Dataset> Dimensions: (time: 28, x: 275, y: 205) Coordinates: * time (time) object 1980-09-16 12:00:00 ... 1982-12-17 00:00:00 xc (y, x) float64 ... yc (y, x) float64 ... Dimensions without coordinates: x, y Data variables: Tair (time, y, x) float64 ... Attributes: title: /workspace/jhamman/processed/R1002RBRxaaa01a/l... institution: U.W. source: RACM R1002RBRxaaa01a output_frequency: daily output_mode: averaged convention: CF-1.4 references: Based on the initial model of Liang et al., 19... comment: Output from the Variable Infiltration Capacity... nco_openmp_thread_number: 1 NCO: "4.6.0" history: Tue Dec 27 14:15:22 2016: ncatted -a dimension... In [5]: import cftime

In [6]: cftime.version
Out[6]: '1.0.3.4'

In [7]: xr.version
Out[7]: '0.14.0' ```