issues

The goal here is to make something extensible that we can live with for quite some time, and to clean up the internals of xarray's backend interface.

Most of these are analogues of existing xarray classes with a cleaned up interface. I have not yet worried about backwards compatibility or tests -- I would appreciate feedback on the approach here.

Several parts of the logic exist for the sake of dask. I've included the word "dask" in comments to facilitate inspection by mrocklin.

CC @rabernat, @pwolfram, @jhamman, @mrocklin -- for review

CC @mcgibbon, @JoyMonteiro -- this is relevant to our discussion today about adding support for appending to netCDF files. Don't let this stop you from getting started on that with the existing interface, though.

This is an internal refactor of reindexing/alignment to use Variable.getitem_with_mask.

As noted back in https://github.com/pydata/xarray/pull/1751#issuecomment-348380756, there is a nice improvement for alignment with dask (~100x improvement) but we are slower in several cases with NumPy (2-3x).

ASV results (smaller ratio is better): before after ratio [e31cf43e] [5830f2f8] 4.85ms 4.86ms 1.00 reindexing.Reindex.time_1d_coarse 98.15ms 98.97ms 1.01 reindexing.Reindex.time_1d_fine_all_found + 96.88ms 210.71ms 2.17 reindexing.Reindex.time_1d_fine_some_missing 24.47ms 25.18ms 1.03 reindexing.Reindex.time_2d_coarse 433.26ms 437.19ms 1.01 reindexing.Reindex.time_2d_fine_all_found + 245.20ms 711.36ms 2.90 reindexing.Reindex.time_2d_fine_some_missing - 23.78ms 12.79ms 0.54 reindexing.Reindex.time_reindex_coarse - 409.89ms 230.75ms 0.56 reindexing.Reindex.time_reindex_fine_all_found + 233.41ms 369.48ms 1.58 reindexing.Reindex.time_reindex_fine_some_missing 14.39ms 14.20ms 0.99 reindexing.ReindexDask.time_1d_coarse 184.07ms 182.64ms 0.99 reindexing.ReindexDask.time_1d_fine_all_found - 1.44s 277.03ms 0.19 reindexing.ReindexDask.time_1d_fine_some_missing 95.49ms 94.49ms 0.99 reindexing.ReindexDask.time_2d_coarse 910.11ms 916.47ms 1.01 reindexing.ReindexDask.time_2d_fine_all_found failed 997.33ms n/a reindexing.ReindexDask.time_2d_fine_some_missing Note that reindexing.ReindexDask.time_2d_fine_some_missing timed out previously, which I think indicates that it took longer than 60 seconds.

• [x] Tests passed (for all non-documentation changes)
• [x] Passes git diff upstream/master **/*py | flake8 --diff (remove if you did not edit any Python files)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Fixes #1172

The serializable lock will be useful for dask.distributed or multi-processing (xref #798, #1173, among others).

Fixes #624

New shift method for shifting datasets or arrays along a dimension:

``` In [1]: import xray

In [2]: array = xray.DataArray([5, 6, 7, 8], dims='x')

In [3]: array.shift(x=2) Out[3]: <xray.DataArray (x: 4)> array([ nan, nan, 5., 6.]) Coordinates: * x (x) int64 0 1 2 3 ```

Based on the API proposed for roll in https://github.com/xray/xray/issues/624

This is an alternative to #125 which I think is a little cleaner.

Basically, there was a bug where Variable.values for datetime64 arrays always made a copy of values. This made it impossible to edit variable values in-place.

@akleeman would appreciate your thoughts.