Follow up to https://github.com/pydata/xarray/pull/8586

I believe this was missed in v2023.08.0 (Aug 18, 2023).

xref: https://github.com/conda-forge/xarray-feedstock/pull/97

Add summary methods to Dataset object. For example, it would be great if you could summarize a entire dataset in a single line.

(1) Mean of all variables in dataset.

python mean_ds = ds.mean()

(2) Mean of all variables in dataset along a dimension:

python time_mean_ds = ds.mean(dim='time')

In the case where a dimension is specified and there are variables that don't use that dimension, I'd imagine you would just pass that variable through unchanged.

Related to #122.

• [x] towards understanding #7079
• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

• [x] Closes #7835

Thanks @pierre-manchon for the report!

Previously, the first context manager in this test was closed before accessing the data. This resulted in key errors when trying to access the opened dataset.

• [x] Fixes the Zarr V3 parts of https://github.com/pydata/xarray/issues/7707

This updates our minimum versions based on our 24/18/12 month policy.

Details are shown below.

• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

This PR add's the to_zarr method to Xarray's DataArray objects. This allows users to roundtrip named and unnamed DataArrays to Zarr without having to first convert to a Dataset.

• [x] Closes #7692
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Is your feature request related to a problem?

It would be nice to mimic the behavior of DataArray.to_netcdf for the Zarr backend.

Describe the solution you'd like

This should be possible: python xr.open_dataarray('file.nc').to_zarr('store.zarr')

Describe alternatives you've considered

None.

Additional context

xref DataArray.to_netcdf issue/PR: #915 / #990

• [x] Closes #7686
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Is your feature request related to a problem?

Xarray maintains the encoding of datasets read from most of its supported backend formats (e.g. NetCDF, Zarr, etc.). This is very useful when you want to perfectly roundtrip but it often gets in the way, causing conflicts when writing a modified dataset or when appending to another dataset. Most of the time, the solution is to just remove the encoding from the dataset and continue on. The following code sample is found in a number of issues that reference this problem.

```python for v in list(ds.coords.keys()): if ds.coords[v].dtype == object: ds[v].encoding.clear()

for v in list(ds.variables.keys()):
    if ds[v].dtype == object:
        ds[v].encoding.clear()

```

A sample of issues that show variants of this problem.

• https://github.com/pydata/xarray/issues/3476
• https://github.com/pydata/xarray/issues/3739
• https://github.com/pydata/xarray/issues/4380
• https://github.com/pydata/xarray/issues/5219
• https://github.com/pydata/xarray/issues/5969
• https://github.com/pydata/xarray/issues/6329
• https://github.com/pydata/xarray/issues/6352

Describe the solution you'd like

In many cases, the solution to these problems is to leave the original dataset encoding behind and either use Xarray's default encoding (or the backends default) or to specify one's own encoding options. Both cases would benefit from a convenience method to reset the original encoding. Something like would serve this process:

python ds = xr.open_dataset(...).reset_encoding()

Describe alternatives you've considered

Variations on the API above could also be considered:

python xr.open_dataset(..., keep_encoding=False)

or even: python with xr.set_options(keep_encoding=False): ds = xr.open_dataset(...)

We can/should also do a better job of surfacing inconsistent encoding in our backends (e.g. to_netcdf).

Additional context

No response

• [x] Closes #7624

What is your issue?

Now that we've bumped our minimum NumPy version to 1.21, we can address this comment:

https://github.com/pydata/xarray/blob/b21f62ee37eea3650a58e9ffa3a7c9f4ae83006b/xarray/core/types.py#L57-L62

I decided not to tackle this as part of #7461 but we may be able to do something like this:

python from numpy.typing._dtype_like import _DTypeLikeNested, _ShapeLike, _SupportsDType

xref: #6834 cc @headtr1ck

This switches our primary linter to Ruff. As adervertised, Ruff is very fast. Plust we get the benefit of using a single tool that combines the previous functionality of pyflakes, isort, and pyupgrade.

• [x] Closes https://twitter.com/TEGNicholasCode/status/1613226956887056385
• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

cc @max-sixty, @TomNicholas

This should fix our doc build.

• [x] Closes #7434

This PR finally deprecates the PyNIO backend. PyNIO is technically in maintenance mode but it hasn't had any maintenance in 4+ years. Its conda packages cannot be installed in any of our test environments. I have added a future warning to the NioDataStore.__init__ method and noted the deprecation in the IO docs.

• [x] Closes #4491
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

The min versions checks are failing in #6475. This hopefully fixes those failures.

• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

In light of the new splash page site (https://xarray.dev), this PR updates the documentation site's index page to simply provide pointers to key parts of Xarray's documentation.

TODOs: - [x] Get feedback on the content and layout - [x] Update the Icon SVGs (these along with the layout were borrowed, in part, from Pandas).

cc @andersy005, @rabernat

This adds an encoding option to Xarray's to_dict methods.

• [x] Closes #6634
• [x] Tests added
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [x] New functions/methods are listed in api.rst

Is your feature request related to a problem?

When using Xarray's to_dict methods to record a Dataset's schema, it would be useful to (optionally) include encoding in the output.

Describe the solution you'd like

The feature request may be resolved by simply adding an encoding keyword argument. This may look like this:

python ds = xr.Dataset(...) ds.to_dict(data=False, encoding=True)

Describe alternatives you've considered

It is currently possible to manually extract encoding attributes but this is a less desirable solution.

xref: https://github.com/pangeo-forge/pangeo-forge-recipes/issues/256

With some acrobatics, it is possible to open file-like objects to rasterio. It would be useful if xarray supported this workflow, particularly for working with cloud optimized geotiffs and fs-spec.

MCVE Code Sample

```python with open('my_data.tif', 'rb') as f: da = xr.open_rasterio(f)

```

Expected Output

DataArray -> equivalent to xr.open_rasterio('my_data.tif')

Problem Description

We only currently allow str, rasterio.DatasetReader, or rasterio.WarpedVRT as inputs to open_rasterio.

Versions

xref: https://github.com/pangeo-data/pangeo-datastore/issues/109

This PR updates our step-by-step guide for releasing Xarray. It makes a few minor changes to account for #6206 and officially documents the switch to CALVER. This should be clearly documented in whats-new.rst as part of the first release utilizing CALVER.

Also, note that this should probably wait until we make the 0.20.1 patch release.

• [x] Closes #6176, #6206
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

Xarray is planning to switch to Calendar versioning (calver). This issue serves as a general announcement.

The idea has come up in multiple developer meetings (#4001) and is part of a larger effort to increase our release cadence (#5927). Today's developer meeting included unanimous consent for the change. Other projects in Xarray's ecosystem have also made this change recently (e.g. https://github.com/dask/community/issues/100). While it is likely we will make this change in the next release or two, users and developers should feel free to voice objections here.

The proposed calver implementation follows the same schema as the Dask project, that is; YYYY.MM.X (4 digit year, two digit month, one digit micro zero-indexed version. For example, the code block below provides comparison of the current and future version tags:

```python In [1]: import xarray as xr

current

In [2]: xr.version Out[2]: '0.19.1'

proposed

In [2]: xr.version Out[2]: '2022.01.0' ```

cc @pydata/xarray

We are in the process of moving our documentation url from https://xarray.pydata.org to https://docs.xarray.dev. This PR makes that change throughout the documentation. Additionally, I corrected some broken links and fixed some missing https urls in the process.

cc @andersy005

• [x] Closes #4139
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

cc @scottyhq and @martindurant

xref: https://github.com/pangeo-data/pangeo-datastore/issues/109

This adds a new file to the root of the Xarray repository, CITATION.cff. GitHub recently added support for citation files and adding this file will add a UI feature to the Xarray GitHub repo.

The author list is based on the latest Zenodo release (0.20.1) and I did my best to find everyone's ORCIDs.

The injected apply/reduce methods for the Groupby and Rolling objects are not shown in the api documentation page. While there is obviously a fair bit of overlap between the similar DataArray/Dataset methods, it would help users to know what methods are available to the Groupby and Rolling methods if we explicitly listed them in the documentation. Suggestions on the best format to show these mehtods (e.g. Rolling.mean) are welcomed.

• [x] Passes pre-commit run --all-files

cc @pydata/xarray

In today's dev call, we opted to schedule a separate meeting to discuss the backends refactor that BOpen (@alexamici and his team) is beginning to work on. This issue is meant to coordinate the scheduling of this meeting. To that end, I've created the following Doodle Poll to help choose a time: https://doodle.com/poll/4mtzxncka7gee4mq

Anyone from @pydata/xarray should feel free to join if there is interest. At a minimum, I'm hoping to have @alexamici, @aurghs, @shoyer, and @rabernat there.

Please respond to the poll by COB tomorrow so I can quickly get the meeting on the books. Thanks!

In this PR, I'm experimenting with using the entrypoints package to support 3rd party backends. This does not attempt to solidify the API for what the store is, I feel like that should happen in a second PR. Here's how it would work...

In @rabernat's xmitgcm package, there is a _MDSDataStore that inherits from xarray.backends.common.AbstractDataStore. To allow reading mds datasets directly in xarray.open_dataset, xmitgcm would add the following lines to its setup.py file:

python setup( ... entry_points={ 'xarray.backends': [ 'mds = xmitgcm.mds_store:_MDSDataStore', ... ] } )

Xarray would then be able to discover this backend at runtime and users could use the store directly in open_dataset calls like this:

python ds = xr.open_dataset('./path/to/file.mds', engine='mds', backend_kwargs={...})

Note: I recognize that xmitgcm.open_mdsdataset has a bunch of other user options that I'm likely ignoring here but this is meant just as an illustration.

Now a list of caveats and things to consider:

1. I have only done this for open_dataset, not for to_netcdf. We may want to consider more generic serialization method that allows for plug-able writers.
2. open_dataset has some special handling for some readers (lock and group selection, file-like objects, etc.). We should work toward moving as much of that logic into the Store objects as possible.
3. We should decide what to do when a 3rd party plugin conflicts with an existing backend. For example, someone could include an entrypoint with the key of netcdf4.

• [x] Partially closes #1970
• [ ] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

I have an application where I'd like to use apply_ufunc with dask on a function that requires multiple inputs and outputs. This was left as a TODO item in the #1517. However, its not clear to me looking at the code how this can be done given the current form of dask's atop. I'm hoping @shoyer has already thought of a clever solution here...

Code Sample, a copy-pastable example if possible

```python def func(foo, bar):

assert foo.shape == bar.shape
spam = np.zeros_like(bar)
spam2 = np.full_like(bar, 2)


return spam, spam2

foo = xr.DataArray(np.zeros((10, 10))).chunk() bar = xr.DataArray(np.zeros((10, 10))).chunk() + 5

xrfunc = xr.apply_ufunc(func, foo, bar, output_core_dims=[[], []], dask='parallelized') ```

Problem description

This currently raises a NotImplementedError.

Expected Output

Multiple dask arrays. In my example above, two dask arrays.

Output of xr.show_versions()

cc @mrocklin, @arbennett

• [x] Fixes #3851
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #1850

Conda-forge has dropped support for 32-bit windows builds (https://github.com/conda-forge/cftime-feedstock/issues/2#issuecomment-385485144). Do we want to continue testing against this environment? The point becomes moot after #1876 gets wrapped up in ~7 months.

xref: https://github.com/pydata/xarray/pull/1252

We recently built a prototype REST API for serving xarray datasets via a Fast-API application (see #3850 for more details). In the process of doing this, we needed to use a few internal functions in Xarray's Zarr backend:

python from xarray.backends.zarr import ( _DIMENSION_KEY, _encode_zarr_attr_value, _extract_zarr_variable_encoding, encode_zarr_variable, ) from xarray.core.pycompat import dask_array_type from xarray.util.print_versions import get_sys_info, netcdf_and_hdf5_versions

Obviously, none of these imports are really meant for use outside of Xarray's backends so I'd like to discuss how we may go about exposing these functions (or variables) as semi-public (advanced use) API features. Thoughts?

cc @rabernat

In https://github.com/pydata/xarray/pull/1170#discussion_r94078121, @shoyer writes:

...I would consider deprecating the encoding argument to DataArray instead. It would also make sense to get rid of the compat argument to Dataset.

These extra arguments are not part of the fundamental xarray data model and thus are a little distracting, especially to new users.

@pydata/xarray and others, what do we think about deprecating the compat argument to the Dataset constructor and the encoding arguement to the DataArray (and Dataset via #1170).

We've recently released Xpublish. This PR adds the project to the _related-projects` page in the Xarray documentation. To find out more about Xpublish, check out the docs or the release announcement blogpost.

We have an application where we want to use apply_ufunc to apply a function that takes two 1-D arrays and returns a scalar value (basically a reduction over the only axis). We start with two DataArrays that share all the same dimensions - except for the lengths of the dimension we'll be reducing along (t in this case):

```python def diff_mean(X, y): ''' a function that only works on 1d arrays that are different lengths''' assert X.ndim == 1, X.ndim assert y.ndim == 1, y.ndim assert len(X) != len(y), X return X.mean() - y.mean()

X = np.random.random((10, 4, 5)) y = np.random.random((6, 4, 5))

Xda = xr.DataArray(X, dims=('t', 'x', 'y')).chunk({'t': -1, 'x': 2, 'y': 2}) yda = xr.DataArray(y, dims=('t', 'x', 'y')).chunk({'t': -1, 'x': 2, 'y': 2}) ```

Then, we'd like to use apply_ufunc to apply our function (e.g. diff_mean):

python out = xr.apply_ufunc( diff_mean, Xda, yda, vectorize=True, dask="parallelized", output_dtypes=[np.float], input_core_dims=[['t'], ['t']], )

This fails with an error when aligning the t dimensions:

```python-traceback

ValueError Traceback (most recent call last) <ipython-input-4-e90cf6fba482> in <module> 9 dask="parallelized", 10 output_dtypes=[np.float], ---> 11 input_core_dims=[['t'], ['t']], 12 )

~/miniconda3/envs/xarray-ml/lib/python3.7/site-packages/xarray/core/computation.py in apply_ufunc(func, input_core_dims, output_core_dims, exclude_dims, vectorize, join, dataset_join, dataset_fill_value, keep_attrs, kwargs, dask, output_dtypes, output_sizes, *args) 1042 join=join, 1043 exclude_dims=exclude_dims, -> 1044 keep_attrs=keep_attrs 1045 ) 1046 elif any(isinstance(a, Variable) for a in args):

~/miniconda3/envs/xarray-ml/lib/python3.7/site-packages/xarray/core/computation.py in apply_dataarray_vfunc(func, signature, join, exclude_dims, keep_attrs, *args) 222 if len(args) > 1: 223 args = deep_align( --> 224 args, join=join, copy=False, exclude=exclude_dims, raise_on_invalid=False 225 ) 226

~/miniconda3/envs/xarray-ml/lib/python3.7/site-packages/xarray/core/alignment.py in deep_align(objects, join, copy, indexes, exclude, raise_on_invalid, fill_value) 403 indexes=indexes, 404 exclude=exclude, --> 405 fill_value=fill_value 406 ) 407

~/miniconda3/envs/xarray-ml/lib/python3.7/site-packages/xarray/core/alignment.py in align(join, copy, indexes, exclude, fill_value, *objects) 321 "arguments without labels along dimension %r cannot be " 322 "aligned because they have different dimension sizes: %r" --> 323 % (dim, sizes) 324 ) 325

ValueError: arguments without labels along dimension 't' cannot be aligned because they have different dimension sizes: {10, 6}

```

https://nbviewer.jupyter.org/gist/jhamman/0e52d9bb29f679e26b0878c58bb813d2

I'm curious if this can be made to work with apply_ufunc or if we should pursue other options here. Advice and suggestions appreciated.

Output of xr.show_versions()

• [x] Closes #3563
• [ ] Tests added
• [ ] Passes black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Toward https://github.com/pydata/xarray/issues/3380#issuecomment-539224341
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

See below for inline comments where I could use some input from @shoyer and @crusaderky

3358 is going to make some fairly major changes to the minimum supported versions of required and optional dependencies. We also have a few bug fixes that have landed since releasing 0.13 that would be good to get out.

From what I can tell, the following pending PRs are close enough to get into this release. - [ ] ~tests for arrays with units #3238~ - [x] map_blocks #3276 - [x] Rolling minimum dependency versions policy #3358 - [x] Remove all OrderedDict's (#3389) - [x] Speed up isel and __getitem__ #3375 - [x] Fix concat bug when concatenating unlabeled dimensions. #3362 - [ ] ~Add hypothesis test for netCDF4 roundtrip #3283~ - [x] Fix groupby reduce for dataarray #3338 - [x] Need a fix for https://github.com/pydata/xarray/issues/3377

Am I missing anything else that needs to get in?

I think we should aim to wrap this release up soon (this week). I can volunteer to go through the release steps once we're ready.

• [x] Closes #3377
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #1571
• [x] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #2744
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #1910
• [x] Tests added

xref: #2683

Problem description

Currently on Travis, the Pynio tests are being skipped. The py27-cdat+iris+pynio is supposed to be running tests for each of these but it is not.

https://travis-ci.org/pydata/xarray/jobs/341426116#L2429-L2518

I can't look at this right now in depth but I'm wondering if this is related to #1531.

reported by @WeatherGod

This is passing in my local test environment. We'll see on CI...

• [x] Closes #535
• [x] Tests added

As described in #2683, our test suite is failing on Travis with an unfortunate segfault. For now, I've just taken rasterio (and therefore GDAL) out of the offending environment. I'll use this PR to test a few other options.

cc @max-sixty

• [x] Closes #2683
• [ ] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

This dev build is failing due to problems with bottlenecks setup script. Generally, the bottleneck package seems to be missing some maintenance effort so until a new release is issued, I don't think we need to be testing against its dev state.

• [x] Closes #1109

xref: #2661

• [x] Closes #697
• [x] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

Improved error message for invalid encodings.

• [x] Closes #2728
• [ ] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

I was feeling particularly zealous today so I decided to see what it would take to strip out all the Python 2 compatibility code in xarray. I expect some will feel its too soon to merge this so I'm mostly putting this up for show-and-tell and to highlight some of the knots we've tied ourselves into over the years.

• [x] Closes #1876
• [ ] Tests added
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

Almost all of our unit tests are against the dask's default scheduler (usually dask.threaded). While it is true that beauty of dask is that one can separate the scheduler from the logical implementation, there are a few idiosyncrasies to consider, particularly in xarray's backends. To that end, we have a few tests covering the integration of the distributed scheduler with xarray's backends but the test coverage is not particularly complete.

If nothing more, I think it is worth considering tests that use the threaded, multiprocessing, and distributed schedulers for a larger subset of the backends tests (those that use dask).

Note, I'm bringing this up because I'm seeing some failing tests in #1793 that are unrelated to my code change but do appear to be related to dask and possibly a different different default scheduler (example failure).

This is a WIP. I expect the CI changes to take a few iterations.

• [x] Closes #1876
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

The timeline for dropping Python 2.7 support for new Xarray releases is the end of 2018.

This issue can be used to track the necessary documentation and code changes to make that happen.

xref: #1830

Just adding a full environment for python 3.7.

• [x] Extends #2271
• [x] Tests added

• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

In working on an xarray/dask tutorial, I've come to realize we eagerly load the tutorial datasets in xarray.tutorial.load_dataset. I'm going to just say that I don't think we should do that but I could be missing some rational. I didn't open an issue so please feel free to share thoughts here.

One option would be to create a new function (xr.tutorial.open_dataset) that does what I'm suggesting and then slowly deprecate tutorial.load_dataset. Thoughts?

xref: https://github.com/dask/dask-examples/pull/51

Following up on #2271. This switches the set language in our Travis-CI config from "python" to "generic". Since we don't use any of the Travis Python utilities, we didn't really need the python setting and the generic setting gives a few benefits:

• smaller base image which should give a bit faster spin-up time

• build matrix without reliance on python version, instead we just point to the conda environment file

• [x] Tests passed (for all non-documentation changes)

• [x] Tests added
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Pandas' date_range function is a fast and flexible way to create DateTimeIndex objects. Now that we have a functioning CFTimeIndex, it would be great to add a version of the date_range function that supports other calendars and dates out of range for Pandas.

Code Sampl and expected output

```python In [1]: import xarray as xr

In [2]: xr.date_range('2000-02-26', '2000-03-02') Out[2]: DatetimeIndex(['2000-02-26', '2000-02-27', '2000-02-28', '2000-02-29', '2000-03-01', '2000-03-02'], dtype='datetime64[ns]', freq='D')

In [3]: xr.date_range('2000-02-26', '2000-03-02', calendar='noleap') Out[3]: CFTimeIndex(['2000-02-26', '2000-02-27', '2000-02-28', '2000-03-01', '2000-03-02'], dtype='cftime.datetime', freq='D') ```

• [ ] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes)
• [ ] Tests added (for all bug fixes or enhancements)
• [ ] Tests passed (for all non-documentation changes)
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Xarray is now a fiscally sponsored project of NumFOCUS. This PR adds a few blurbs of text highlighting that on the main readme and index page of the docs.

TODO: - Update flipcause to xarray specific donation page

followup PR fixing two small typos in my previous PR.

• [x] Closes #2278
• [x] Tests added
• [ ] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #2238
• [ ] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] closes #2200, closes #1829, closes #2203
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

This updates the following minimum versions:

• numpy: 1.11 (Mar 27, 2016) --> 1.12 (Jan 15, 2017)
• pandas: 0.18 (Mar 11, 2016) --> 0.19 (Oct 2, 2016)
• dask: 0.9 (May 10, 2016) --> 0.16

and drops our tests for python 3.4.

Python 3.7-final is due out in June (PEP 537). When do we want to deprecate 3.4 and when should we drop support all together. @maxim-lian brought this up in a PR he's working on: https://github.com/pydata/xarray/pull/1828#issuecomment-357562144.

For reference, we dropped Python 3.3 in #1175 (12/20/2016).

Xarray currently states that it supports dask version 0.9 and later. However, 1) I don't think this is true and my quick test shows that some of our tests fail using dask 0.9, and 2) we have a growing number of tests that are being skipped for older dask versions:

$ grep -irn "dask.__version__" xarray/tests/*py xarray/tests/__init__.py:90: if LooseVersion(dask.__version__) < '0.18': xarray/tests/test_computation.py:755: if LooseVersion(dask.__version__) < LooseVersion('0.17.3'): xarray/tests/test_computation.py:841: if not use_dask or LooseVersion(dask.__version__) > LooseVersion('0.17.4'): xarray/tests/test_dask.py:211: @pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:223: @pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:284: @pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:296: @pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:387: if LooseVersion(dask.__version__) == LooseVersion('0.15.3'): xarray/tests/test_dask.py:784: pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:802: pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_dask.py:818:@pytest.mark.skipif(LooseVersion(dask.__version__) <= '0.15.4', xarray/tests/test_variable.py:1664: if LooseVersion(dask.__version__) <= LooseVersion('0.15.1'): xarray/tests/test_variable.py:1670: if LooseVersion(dask.__version__) <= LooseVersion('0.15.1'):

I'd like to see xarray bump the minimum version number of dask to something around 0.15.4 (Oct. 2017) or 0.16 (Nov. 2017).

cc @mrocklin, @pydata/xarray

Pandas is dropping Numpy 1.11 and earlier in their 0.24 release. It is probably easiest to follow suit with xarray.

xref: https://github.com/pandas-dev/pandas/issues/21242

• [x] Closes #2225
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I think the following block of code is more strict than either dask or zarr requires:

https://github.com/pydata/xarray/blob/6c3abedf906482111b06207b9016ea8493c42713/xarray/backends/zarr.py#L80-L89

It should be possible to have uneven chunks in the last position of multiple dimensions in a zarr dataset.

Code Sample, a copy-pastable example if possible

```python In [1]: import xarray as xr

In [2]: import dask.array as dsa

In [3]: da = xr.DataArray(dsa.random.random((8, 7, 11), chunks=(3, 3, 3)), dims=('x', 'y', 't'))

In [4]: da Out[4]: <xarray.DataArray 'da.random.random_sample-1aed3ea2f9dd784ec947cb119459fa56' (x: 8, y: 7, t: 11)> dask.array<shape=(8, 7, 11), dtype=float64, chunksize=(3, 3, 3)> Dimensions without coordinates: x, y, t

In [5]: da.data.chunks Out[5]: ((3, 3, 2), (3, 3, 1), (3, 3, 3, 2))

In [6]: da.to_dataset('varname').to_zarr('/Users/jhamman/workdir/test_chunks.zarr') /Users/jhamman/anaconda/bin/ipython:1: FutureWarning: the order of the arguments on DataArray.to_dataset has changed; you now need to supply name as a keyword argument #!/Users/jhamman/anaconda/bin/python

ValueError Traceback (most recent call last) <ipython-input-7-32fa9a7d0276> in <module>() ----> 1 da.to_dataset('varname').to_zarr('/Users/jhamman/workdir/test_chunks.zarr')

~/anaconda/lib/python3.6/site-packages/xarray/core/dataset.py in to_zarr(self, store, mode, synchronizer, group, encoding, compute) 1185 from ..backends.api import to_zarr 1186 return to_zarr(self, store=store, mode=mode, synchronizer=synchronizer, -> 1187 group=group, encoding=encoding, compute=compute) 1188 1189 def unicode(self):

~/anaconda/lib/python3.6/site-packages/xarray/backends/api.py in to_zarr(dataset, store, mode, synchronizer, group, encoding, compute) 856 # I think zarr stores should always be sync'd immediately 857 # TODO: figure out how to properly handle unlimited_dims --> 858 dataset.dump_to_store(store, sync=True, encoding=encoding, compute=compute) 859 860 if not compute:

~/anaconda/lib/python3.6/site-packages/xarray/core/dataset.py in dump_to_store(self, store, encoder, sync, encoding, unlimited_dims, compute) 1073 1074 store.store(variables, attrs, check_encoding, -> 1075 unlimited_dims=unlimited_dims) 1076 if sync: 1077 store.sync(compute=compute)

~/anaconda/lib/python3.6/site-packages/xarray/backends/zarr.py in store(self, variables, attributes, args, kwargs) 341 def store(self, variables, attributes, args, kwargs): 342 AbstractWritableDataStore.store(self, variables, attributes, --> 343 *args, kwargs) 344 345 def sync(self, compute=True):

~/anaconda/lib/python3.6/site-packages/xarray/backends/common.py in store(self, variables, attributes, check_encoding_set, unlimited_dims) 366 self.set_dimensions(variables, unlimited_dims=unlimited_dims) 367 self.set_variables(variables, check_encoding_set, --> 368 unlimited_dims=unlimited_dims) 369 370 def set_attributes(self, attributes):

~/anaconda/lib/python3.6/site-packages/xarray/backends/common.py in set_variables(self, variables, check_encoding_set, unlimited_dims) 403 check = vn in check_encoding_set 404 target, source = self.prepare_variable( --> 405 name, v, check, unlimited_dims=unlimited_dims) 406 407 self.writer.add(source, target)

~/anaconda/lib/python3.6/site-packages/xarray/backends/zarr.py in prepare_variable(self, name, variable, check_encoding, unlimited_dims) 325 326 encoding = _extract_zarr_variable_encoding( --> 327 variable, raise_on_invalid=check_encoding) 328 329 encoded_attrs = OrderedDict()

~/anaconda/lib/python3.6/site-packages/xarray/backends/zarr.py in _extract_zarr_variable_encoding(variable, raise_on_invalid) 181 182 chunks = _determine_zarr_chunks(encoding.get('chunks'), variable.chunks, --> 183 variable.ndim) 184 encoding['chunks'] = chunks 185 return encoding

~/anaconda/lib/python3.6/site-packages/xarray/backends/zarr.py in _determine_zarr_chunks(enc_chunks, var_chunks, ndim) 87 "Zarr requires uniform chunk sizes excpet for final chunk." 88 " Variable %r has incompatible chunks. Consider " ---> 89 "rechunking using chunk()." % (var_chunks,)) 90 # last chunk is allowed to be smaller 91 last_var_chunk = all_var_chunks[-1]

ValueError: Zarr requires uniform chunk sizes excpet for final chunk. Variable ((3, 3, 2), (3, 3, 1), (3, 3, 3, 2)) has incompatible chunks. Consider rechunking using chunk(). ```

Problem description

[this should explain why the current behavior is a problem and why the expected output is a better solution.]

Expected Output

IIUC, Zarr allows multiple dims to have uneven chunks, so long as they are all in the last position:

```Python In [9]: import zarr

In [10]: z = zarr.zeros((8, 7, 11), chunks=(3, 3, 3), dtype='i4')

In [11]: z.chunks Out[11]: (3, 3, 3) ```

Output of xr.show_versions()

• [x] Closes #2121
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

cc @rsignell-usgs

Code Sample, a copy-pastable example if possible

```Python In [1]: import xarray as xr

In [2]: ds = xr.open_rasterio('RGB.byte.tif')

In [3]: ds Out[3]: <xarray.DataArray (band: 3, y: 718, x: 791)> [1703814 values with dtype=uint8] Coordinates: * band (band) int64 1 2 3 * y (y) float64 2.827e+06 2.826e+06 2.826e+06 2.826e+06 2.826e+06 ... * x (x) float64 1.021e+05 1.024e+05 1.027e+05 1.03e+05 1.033e+05 ... Attributes: transform: (101985.0, 300.0379266750948, 0.0, 2826915.0, 0.0, -300.0417... crs: +init=epsg:32618 res: (300.0379266750948, 300.041782729805) is_tiled: 0 nodatavals: (0.0, 0.0, 0.0)

In [4]: import pickle

In [5]: pickle.dumps(ds)

TypeError Traceback (most recent call last) <ipython-input-5-a165c2473431> in <module>() ----> 1 pickle.dumps(ds)

TypeError: can't pickle rasterio._io.RasterReader objects ```

Problem description

Originally reported by @rsignell-usgs in https://github.com/pangeo-data/pangeo/issues/249#issuecomment-388445370, the rasterio backend is not pickle-able. This obviously causes problems when using dask-distributed. We probably need to use DataStorePickleMixin or something similar on rasterio datasets to allow multiple readers of the same dataset.

Expected Output

python pickle.dumps(ds)

returns a pickled dataset.

Output of xr.show_versions()

• [x] Closes #2134
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

• [x] Closes #1300 (in a more portable way)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

This eliminates the need to edit setup.py before / after release and is a nice step towards simplifying xarray's release process.

• [x] Closes #2140 ~- [ ] Tests added (for all bug fixes or enhancements)~
• [ ] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

cc @spencerkclark and @shoyer

review of this can wait until after #1800 is merged.

• [x] Closes #1784
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

cc @mrocklin

• [x] Closes #1981
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I'm sharing this in the hopes of getting comments from @mrocklin and @pydata/xarray.

What this does:

• implements a dask.bag map/apply on the xarray open_dataset and preprocess steps in open_mfdataset
• adds a new parallel option to open_mfdataset
• provides about a 40% speedup in opening a multifile dataset when using the distributed scheduler (I tested on 1000 netcdf files that took about 9 seconds to open/concatenate in the default configuration)

What it does not do (yet):

• check that autoclose=True when multiple processes are being use (multiprocessing/distributed scheduler)
• provide any speedup with the multiprocessing backend (I do not understand why this is)

Benchmark Example

```Python In [1]: import xarray as xr ...: import dask ...: import dask.threaded ...: import dask.multiprocessing ...: from dask.distributed import Client ...:

In [2]: c = Client() ...: c ...: Out[2]: <Client: scheduler='tcp://127.0.0.1:59576' processes=4 cores=4>

In [4]: %%time ...: with dask.set_options(get=dask.multiprocessing.get): ...: ds = xr.open_mfdataset('../test_files/test_netcdf_*nc', autoclose=True, parallel=True) ...: CPU times: user 4.76 s, sys: 201 ms, total: 4.96 s Wall time: 7.74 s

In [5]: %%time ...: with dask.set_options(get=c.get): ...: ds = xr.open_mfdataset('../test_files/test_netcdf_*nc', autoclose=True, parallel=True) ...: ...: CPU times: user 1.88 s, sys: 60.6 ms, total: 1.94 s Wall time: 4.41 s

In [6]: %%time ...: with dask.set_options(get=dask.threaded.get): ...: ds = xr.open_mfdataset('../test_files/test_netcdf_*nc') ...: CPU times: user 7.77 s, sys: 247 ms, total: 8.02 s Wall time: 8.17 s

In [7]: %%time ...: with dask.set_options(get=dask.threaded.get): ...: ds = xr.open_mfdataset('../test_files/test_netcdf_*nc', autoclose=True) ...: ...: CPU times: user 7.89 s, sys: 202 ms, total: 8.09 s Wall time: 8.21 s

In [8]: ds Out[8]: <xarray.Dataset> Dimensions: (lat: 45, lon: 90, time: 1000) Coordinates: * lon (lon) float64 0.0 4.045 8.09 12.13 16.18 20.22 24.27 28.31 ... * lat (lat) float64 -90.0 -85.91 -81.82 -77.73 -73.64 -69.55 -65.45 ... * time (time) datetime64[ns] 1970-01-01 1970-01-02 1970-01-11 ... Data variables: foo (time, lon, lat) float64 dask.array<shape=(1000, 90, 45), chunksize=(1, 90, 45)> bar (time, lon, lat) float64 dask.array<shape=(1000, 90, 45), chunksize=(1, 90, 45)> baz (time, lon, lat) float32 dask.array<shape=(1000, 90, 45), chunksize=(1, 90, 45)> Attributes: history: created for xarray benchmarking

```

Code Sample, a copy-pastable example if possible

python xr.open_mfdataset('path/to/many/files*.nc', method='parallel')

Problem description

We have many issues describing the less than stelar performance of open_mfdataset (e.g. #511, #893, #1385, #1788, #1823). The problem can be broken into three pieces: 1) open each file, 2) decode/preprocess each datasets, and 3) merge/combine/concat the collection of datasets. We can perform (1) and (2) in parallel (performance improvements to (3) would be a separate task). Lately, I'm finding that for large numbers of files, it can take many seconds to many minutes just to open all the files in a multi-file dataset of mine.

I'm proposing that we use something like dask.bag to parallelize steps (1) and (2). I've played around with this a bit and it "works" almost right out of the box, provided you are using the "autoclose=True" option. A concrete example:

We could change the line: Python datasets = [open_dataset(p, **open_kwargs) for p in paths] to Python import dask.bag as db paths_bag = db.from_sequence(paths) datasets = paths_bag.map(open_dataset, **open_kwargs).compute()

I'm curious what others think of this idea and what the potential downfalls may be.

• [x] Closes #1464
• [x] Tests added
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Right now, I've just modified the dask distributed integration tests so we can all see the failing tests.

I'm happy to push this further but I thought I'd see if either @shoyer or @mrocklin have an idea where to start?

• [x] Closes #1979 and #1955
• [x] Tests added
• [x] Tests passed

• [x] Closes #1920
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented: see #1920

Code Sample, a copy-pastable example if possible

```python import numpy as np import xarray as xr n_times = 4; n_lats = 10; n_lons = 15 n_points = 4

ds = xr.Dataset({'test_var': (['time', 'latitude', 'longitude'], np.random.random((n_times, n_lats, n_lons)))}) ds.to_netcdf('test.nc')

rand_lons = xr.Variable('points', np.random.randint(0, high=n_lons, size=n_points)) rand_lats = xr.Variable('points', np.random.randint(0, high=n_lats, size=n_points))

ds = xr.open_dataset('test.nc', cache=False) points = ds['test_var'][:, rand_lats, rand_lons] yields:

NotImplementedError Traceback (most recent call last) <ipython-input-7-f16e4cae9456> in <module>() 12 13 ds = xr.open_dataset('test.nc', cache=False) ---> 14 points = ds['test_var'][:, rand_lats, rand_lons]

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/dataarray.py in getitem(self, key) 478 else: 479 # xarray-style array indexing --> 480 return self.isel(**self._item_key_to_dict(key)) 481 482 def setitem(self, key, value):

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/dataarray.py in isel(self, drop, indexers) 759 DataArray.sel 760 """ --> 761 ds = self._to_temp_dataset().isel(drop=drop, indexers) 762 return self._from_temp_dataset(ds) 763

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/dataset.py in isel(self, drop, indexers) 1390 for name, var in iteritems(self._variables): 1391 var_indexers = {k: v for k, v in indexers_list if k in var.dims} -> 1392 new_var = var.isel(var_indexers) 1393 if not (drop and name in var_indexers): 1394 variables[name] = new_var

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/variable.py in isel(self, **indexers) 851 if dim in indexers: 852 key[i] = indexers[dim] --> 853 return self[tuple(key)] 854 855 def squeeze(self, dim=None):

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/variable.py in getitem(self, key) 620 """ 621 dims, indexer, new_order = self._broadcast_indexes(key) --> 622 data = as_indexable(self._data)[indexer] 623 if new_order: 624 data = np.moveaxis(data, range(len(new_order)), new_order)

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/indexing.py in getitem(self, key) 554 555 def getitem(self, key): --> 556 return type(self)(_wrap_numpy_scalars(self.array[key])) 557 558 def setitem(self, key, value):

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/indexing.py in getitem(self, indexer) 521 522 def getitem(self, indexer): --> 523 return type(self)(self.array, self._updated_key(indexer)) 524 525 def setitem(self, key, value):

~/anaconda/envs/pangeo/lib/python3.6/site-packages/xarray/core/indexing.py in _updated_key(self, new_key) 491 'Vectorized indexing for {} is not implemented. Load your ' 492 'data first with .load() or .compute(), or disable caching by ' --> 493 'setting cache=False in open_dataset.'.format(type(self))) 494 495 iter_new_key = iter(expanded_indexer(new_key.tuple, self.ndim))

NotImplementedError: Vectorized indexing for <class 'xarray.core.indexing.LazilyIndexedArray'> is not implemented. Load your data first with .load() or .compute(), or disable caching by setting cache=False in open_dataset. ```

Problem description

Raising a NotImplementedError here is fine but it instructs the user to "disable caching by setting cache=False in open_dataset" which I've already done. So my questions are 1) should we expect this to work and 2) if not

Expected Output

Ideally, we can get the same behavior as:

```python ds = xr.open_dataset('test2.nc', cache=False).load() points = ds['test_var'][:, rand_lats, rand_lons]

<xarray.DataArray 'test_var' (time: 4, points: 4)> array([[0.939469, 0.406885, 0.939469, 0.759075], [0.470116, 0.585546, 0.470116, 0.37833 ], [0.274321, 0.648218, 0.274321, 0.383391], [0.754121, 0.078878, 0.754121, 0.903788]]) Dimensions without coordinates: time, points ```

without needing to use .load()

Output of xr.show_versions()

We're close to a minor/bug-fix release (0.10.1). What do we need to get done before that can happen?

• [x] #1800 Performance improvements to Zarr (@jhamman)
• [ ] #1793 Fix for to_netcdf writes with dask-distributed (@jhamman, could use help)
• [x] #1819 Normalisation for RGB imshow

Help wanted / bugs that no-one is working on: - [ ] #1792 Comparison to masked numpy arrays - [ ] #1764 groupby_bins fails for empty bins

What else?

• [x] Closes #1937
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Thanks @fujiisoup for the report.

• [x] Closes #640
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This is something we've talked about for a while and I'm capitalizing on a moment of inspiration. Full disclosure, I've taken most of this from Pandas and edited it just where it makes sense for Xarray.

If others would like specific changes to this, please comment only on doc/contributing.rst, ~CONTRIBUTING.md is auto generated with pandoc~. @pydata/xarray, feel free to push directly to this branch if there are larger edits you'd like to add.

• [x] Helps with #1084
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

I've added a temporary travis build with the master branch of netcdftime. After a while, we can probably remove that.

This is helping us move towards https://github.com/Unidata/netcdf4-python/issues/601 and #1252

cc @jswhit and @spencerkclark

• [x] Closes #1894
• [ ] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

cc @mrocklin

• [x] Closes #1898 (remove if there is no corresponding issue, which should only be the case for minor changes)
• [x] Tests added (for all bug fixes or enhancements)
• [ ] Tests passed (for all non-documentation changes)
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

xray's unit testing system uses Python's standard unittest framework. pytest offers a more flexible framework requiring less boilerplate code. I recently (#638) introduced pytest into xray's CI builds. This issue proposes incrementally migrating and simplifying xray's unit testing framework to pytest.

• [x] Closes #1830 (remove if there is no corresponding issue, which should only be the case for minor changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

When do we want to drop Python 2 support for Xarray. For reference, Pandas has a stated drop date for Python 2 of the end of 2018 (this year) and Numpy is slightly later and includes an incremental depreciation, final on Jan. 1, 2020.

We may also consider signing this pledge to help make it clear when/why we're dropping Python 2 support: http://www.python3statement.org/

xref: https://github.com/pandas-dev/pandas/issues/18894, https://github.com/numpy/numpy/pull/10006, https://github.com/python3statement/python3statement.github.io/issues/11

• [x] Closes #1867
• [x] Tests passed (for all non-documentation changes)

• [x] Closes #https://github.com/pangeo-data/pangeo/issues/48
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Passes git diff upstream/master **/*py | flake8 --diff (remove if you did not edit any Python files)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

This is building on top of #1799. Based on the suggestion from @alimanfoo in https://github.com/pangeo-data/pangeo/issues/48#issuecomment-353807691, I have reworked the handling of attributes in the zarr backend. There is more to do here, particularly in the set_dimensions arena but this is giving almost a 2x speedup in writing to GCP.

cc @rabernat, @mrocklin and @alimanfoo

We're getting a single test failure in the plot tests on master (link to travis failure. I haven't been able to reproduce this locally yet so I'm just going to post here to see if anyone has any ideas.

Code Sample

```python ___ TestDatetimePlot.test_datetime_line_plot _____ self = <xarray.tests.test_plot.TestDatetimePlot testMethod=test_datetime_line_plot> def test_datetime_line_plot(self): # test if line plot raises no Exception

  self.darray.plot.line()

xarray/tests/test_plot.py:1333:

xarray/plot/plot.py:328: in line return line(self._da, args, *kwargs) xarray/plot/plot.py:223: in line _ensure_plottable(x)

args = (<xarray.DataArray 'time' (time: 12)> array([datetime.datetime(2017, 1, 1, 0, 0), datetime.datetime(2017, 2, 1,... 12, 1, 0, 0)], dtype=object) Coordinates: * time (time) object 2017-01-01 2017-02-01 2017-03-01 2017-04-01 ...,) numpy_types = [<class 'numpy.floating'>, <class 'numpy.integer'>, <class 'numpy.timedelta64'>, <class 'numpy.datetime64'>] other_types = [<class 'datetime.datetime'>] x = <xarray.DataArray 'time' (time: 12)> array([datetime.datetime(2017, 1, 1, 0, 0), datetime.datetime(2017, 2, 1, ...7, 12, 1, 0, 0)], dtype=object) Coordinates: * time (time) object 2017-01-01 2017-02-01 2017-03-01 2017-04-01 ... def _ensure_plottable(*args): """ Raise exception if there is anything in args that can't be plotted on an axis. """ numpy_types = [np.floating, np.integer, np.timedelta64, np.datetime64] other_types = [datetime]

    for x in args:
        if not (_valid_numpy_subdtype(np.array(x), numpy_types)
                or _valid_other_type(np.array(x), other_types)):

          raise TypeError('Plotting requires coordinates to be numeric '
                            'or dates.')

E TypeError: Plotting requires coordinates to be numeric or dates. xarray/plot/plot.py:57: TypeError ```

Expected Output

This test was previously passing

Output of xr.show_versions()

https://travis-ci.org/pydata/xarray/jobs/326640013#L1262

• [x] Closes #1813 ~[ ] Tests added~
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

• [x] Closes #1631
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Rough draft of interpolate method for filling of arbitrary nans.

cc @darothen

I've found a few issues where xarray's new resample / upsample functionality is diverging from Pandas. I think they are mostly surrounding how NaNs are treated. Thoughts from @shoyer, @darothen and others.

Gist with all the juicy details: https://gist.github.com/jhamman/354f0e5ff32a39550ffd25800e7214fc#file-xarray_resample-ipynb

xref: #1608, #1272

• [x] Closes #1798
• [ ] Tests added (ideas for how to test that load is not called? Regression tests from #1609 are passing)
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff (remove if you did not edit any Python files)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API