issues

Intended to fix https://github.com/pydata/xarray/issues/4449

The goal is to restore behavior to match what we had prior to https://github.com/pydata/xarray/pull/4379 for all types of data other than np.ndarray objects

Needs tests!

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] Passes isort . && black . && mypy . && flake8
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

Some utility functions that should be useful for https://github.com/pydata/xarray/issues/1603

Still very much a work in progress -- it would be great if someone has time to finish writing any of these in another PR!

When I read this, I thought it was referring to scattering data somehow :).

Fixes https://github.com/pydata/xarray/issues/3009

Previously, indexing an ImplicitToExplicitIndexingAdapter object could directly return an ExplicitlyIndexed object, which could not be indexed normally, e.g., x[index] could result in an object that could not be indexed properly. This resulted in broken behavior with dask's new _meta attribute.

I'm pretty sure this fix is appropriate, but it does introduce two failing tests with xarray on dask master. In particular, there are now errors raised inside two tests from dask's blockwise_meta helper function: ```

  return meta.astype(dtype)

E AttributeError: 'ImplicitToExplicitIndexingAdapter' object has no attribute 'astype' ```

cc @mrocklin @pentschev

• [x] Tests added

Remove the optional condaforge-rc, netcdf4-dev and pynio-dev builds. These have been continuously failing (due to broken installs), so we shouldn't waste time/energy running them.

• [x] Closes #2149 (remove if there is no corresponding issue, which should only be the case for minor changes)
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

@crusaderky please take a look. Testing here is not as thorough as in https://github.com/pydata/xarray/pull/2150 yet, but it does include a regression test.

It looks like this got inadvertently removed with the flake8 fix in #1925.

• [x] Closes #1944

Building off of discussion in #1087, I would like to propose refactoring xarray.conventions to use an interface based on VariableCoder objects with encode() and decode() methods.

The idea is make it easier to write new backends, by making decoding variables according to CF conventions as simple as calling decode() on each coder in a list of coders, with encoding defined by calling the same list of encoders in the opposite order.

As a proof of concept, here I implement only a single Coder. In addition to making use of xarray's existing lazy indexing behavior, I have written it so that dask arrays are decoded using dask (which would solve #1372)

Eventually, we should port all the coders in xarray.conventions to this new format. This is probably best saved for future PRs -- help would be appreciated!

• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff

• [x] Closes #1232
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

They are all now called out explicitly in the README as well.

• [x] closes #1254
• [x] passes git diff upstream/master | flake8 --diff

Fixes #242

This is useful for getting rid of extraneous scalar variables that arise from indexing, and in particular will resolve an issue for optional indexes: https://github.com/pydata/xarray/pull/1017#issuecomment-260777664

It works!

To preview, see http://xray.readthedocs.org/en/rtd-conda/

Fixes #602

This is a renaming and update of the existing xray.broadcast_arrays function, which now works properly in the light of #648.

xref #649 cc @rabernat

Examples

Broadcast two data arrays against one another to fill out their dimensions:

```

a = xray.DataArray([1, 2, 3], dims='x') b = xray.DataArray([5, 6], dims='y') a <xray.DataArray (x: 3)> array([1, 2, 3]) Coordinates: * x (x) int64 0 1 2 b <xray.DataArray (y: 2)> array([5, 6]) Coordinates: * y (y) int64 0 1 a2, b2 = xray.broadcast(a, b) a2 <xray.DataArray (x: 3, y: 2)> array([[1, 1], [2, 2], [3, 3]]) Coordinates: * x (x) int64 0 1 2 * y (y) int64 0 1 b2 <xray.DataArray (x: 3, y: 2)> array([[5, 6], [5, 6], [5, 6]]) Coordinates: * y (y) int64 0 1 * x (x) int64 0 1 2 ```

Fill out the dimensions of all data variables in a dataset:

```

ds = xray.Dataset({'a': a, 'b': b}) ds2, = xray.broadcast(ds) # use tuple unpacking to extract one dataset ds2 <xray.Dataset> Dimensions: (x: 3, y: 2) Coordinates: * x (x) int64 0 1 2 * y (y) int64 0 1 Data variables: a (x, y) int64 1 1 2 2 3 3 b (x, y) int64 5 6 5 6 5 6 ```

We were using some internal routines in pandas to convert object of datetime objects arrays to datetime64. Predictably, these internal routines have now changed, breaking xray.

This is definitely my fault but also bad luck -- I had a guard against the internal function dissappearing, but not against the keyword arguments changing.

In any case, this fix ensures forwards compatibility with the next release of pandas, which will be coming out next week.

Fixes #553

Also ensures we skip NaN when aggregating with complex dtypes.

~~Still needs release notes.~~

Accessing data from remote datasets now has retrying logic (with exponential backoff) that should make it robust to occasional bad responses from DAP servers.

This provides an interface for re-indexing a dataset or dataset array using the coordinates from another object. Missing values along any coordinate are replaced by NaN.

This method is directly based on the pandas method DataFrame.reindex_like (and the related series and panel variants). Eventually, I would like to build upon this functionality to add a join method to xray.align with the possible values {'outer', 'inner', 'left', 'right'}, just like DataFrame.align.

This PR depends on PR #71, since I use its improved copy method for datasets.

Thanks to some code borrowed from pandas, setup.py now reports the development version of xray as something like "0.1.0.dev-de28cd6".

I also took this opportunity to add xray.version.

This lets us use NaNs instead of masked arrays to indicate missing values.