• [x] Closes #3868
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

Now we use a tuple of indexes for DataArray.pad and Dataset.pad.

• [x] Closes #3562
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Now 0d-DataArray can be used for indexing.

• [x] Closes #2007, #2011
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Added some boundary options for rolling.construct. Currently, the option names are inherited from np.pad, ['edge' | 'reflect' | 'symmetric' | 'wrap']. Do we want a more intuitive name, such as periodic?

• [x] Closes #4328
• [x] Tests added
• [x] Passes isort . && black . && mypy . && flake8

There was a bug in rolling.__repr__ but it was not tested. Fixed and tests are added.

• [x] Closes #4196
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

I noticed that the implementation of nd-rolling is straightforward. The core part is implemented but I am wondering what the best API is, with keeping it backward-compatible.

Obviously, it is basically should look like python da.rolling(x=3, y=3).mean()

A problem is other parameters, centers and min_periods. In principle, they can depend on dimension. For example, we can have center=True only for x but not for y.

So, maybe we allow dictionary for them? python da.rolling(x=3, y=3, center={'x': True, 'y': False}, min_periods={'x': 1, 'y': None}).mean()

The same thing happens for .construct method. python da.rolling(x=3, y=3).construct(x='x_window', y='y_window', stride={'x': 2, 'y': 1}) I'm afraid if this dictionary argument was a bit too redundant.

Does anyone have another idea?

• [x] Closes #1388
• [x] Tests added / passed
• [x] Passes git diff master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

With this PR, ValueError raises if argmin() is called by a multi-dimensional array.

argmin_indexes() method is also added for xr.DataArray. Current API design for argmin_indexes() returns the argmin-indexes as an OrderedDict of DataArrays.

Example: ```python

In [1]: import xarray as xr ...: da = xr.DataArray([[1, 2], [-1, 40], [5, 6]], ...: [('x', ['c', 'b', 'a']), ('y', [1, 0])]) ...: ...: da.argmin_indexes() ...: Out[1]: OrderedDict([('x', <xarray.DataArray 'x' ()> array(1)), ('y', <xarray.DataArray 'y' ()> array(0))])

In [2]: da.argmin_indexes(dims='y') Out[2]: OrderedDict([('y', <xarray.DataArray 'y' (x: 3)> array([0, 0, 0]) Coordinates: * x (x) <U1 'c' 'b' 'a')])

```

(Because the returned object is an OrderedDict, it is not beautifully printed. The returned type can be a xr.Dataset if we want.)

Although in #1388 argmin_indexes() was originally suggested so that we can pass the result into isel_point,

python da.isel_points(**da.argmin_indexes()) current implementation of isel_points does NOT work for this case.

This is mainly because 1. isel_points currently does not work for 0-dimensional or multi-dimensional input. 2. Even for 1-dimensional input (the second one in the above examples), we should also pass x as an indexer rather than the coordinate of indexer.

For 1, I have prepared modification of isel_points to accept multi-dimensional arrays, but I guess it should be in another PR after the API decision. (It is related in #475, and #974.)

For 2, we should either + change API of argmin_indexes to return not only the indicated dimension but also all the dimensions, like

```python In [2]: da.argmin_indexes(dims='y') Out[2]: OrderedDict([('y', array([0, 0, 0]), 'x', array(['c' 'b' 'a']))

`` or + change API ofisel_pointso that it takes care of theindexer's coordinate ifxr.DataArrayis passed for asindexers`.

I originally worked with the second option for the modification of isel_points, the second option breaks the backward-comaptibility and is somehow magical.

Another alternertive is to + change API of argmin_indexes to return xr.Dataset rather than an OrderedDict, and also change API of isel_points to accept xr.Dataset.
It keeps backward-compatibility.

Any comments are welcome.

• [x] Closes #2223
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Now n-dimensional interp works sequentially if possible. It may speed up some cases.

• [x] Closes #4024
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Now it looks like

I'm pretty sure that this workaround is not the best (maybe the second worst), as it only supports the dark mode of vscode but not other environments.

I couldn't find a good way to make a workaround for the general dark-mode. Any advice is welcome.

• [x] Closes #3951
• [x] Tests added
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

It looks dataframe.set_index(index).index == index is not always true.

Added a workaround for this...

• [x] Closes #3669, #3674
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

It is a bit surprising that no members have used xarray with CategoricalIndex... If there is anything missing additionally, please feel free to point it out.

• [x] Closes #3518
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Added sparse option to reindex and unstack. I just added a minimal set of codes necessary to unstack and reindex.

There is still a lot of space to complete the sparse support as discussed in #3245.

• [x] Closes #3518
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Added an option fill_value for unstack. I am trying to add sparse option too, but it may take longer. Probably better to do in a separate PR?

• [x] Closes #3512
• [x] Tests added
• [x] Passes black . && mypy . && flake8
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

There was a bug in set_index, where an old dimension was not updated if it becomes a level of MultiIndex.

• [x] Closes #1775 (remove if there is no corresponding issue, which should only be the case for minor changes)
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Passes git diff upstream/master **/*py | flake8 --diff (remove if you did not edit any Python files)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

I am trying to fix #1775, but tests are still failing. Any help would be appreciated.

• [x] Closes #2940
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fix for #2940 It looks that there was a bug in the previous logic, but I am not sure why it was working...

• [x] Closes #2667
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Started to fixing #2667

• [x] Closes #1288
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I would like to add integrate, which is essentially an xarray-version of np.trapz. I know there was variety of discussions in #1288, but I think it would be nice to limit us within that numpy provides by np.trapz, i.e., 1. only for trapz not rectangle or simps 2. do not care np.nan 3. do not support bounds Most of them (except for 1) can be solved by combining several existing methods.

• [x] Closes #1408
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

[Edit for more clarity] I restarted a new branch to fix #1408 (I closed the older one #1412).

Because the changes I made is relatively large, here I summarize this PR.

Sumamry

In this PR, I newly added two kinds of levels in MultiIndex, index-level and scalar-level. index-level is an ordinary level in MultiIndex (as in current implementation), while scalar-level indicates dropped level (which is newly added in this PR).

Changes in behaviors.

1. Indexing a scalar at a particular level changes that level to scalar-level instead of dropping that level (changed from #767).
2. Indexing a scalar from a MultiIndex, the selected value now becomes a MultiIndex-scalar rather than a scalar of tuple.
3. Enabled indexing along a index-level if the MultiIndex has only a single index-level.

Examples of the output are shown below. Any suggestions for these behaviors are welcome.

```python In [1]: import numpy as np ...: import xarray as xr ...: ...: ds1 = xr.Dataset({'foo': (('x',), [1, 2, 3])}, {'x': [1, 2, 3], 'y': 'a'}) ...: ds2 = xr.Dataset({'foo': (('x',), [4, 5, 6])}, {'x': [1, 2, 3], 'y': 'b'}) ...: # example data ...: ds = xr.concat([ds1, ds2], dim='y').stack(yx=['y', 'x']) ...: ds Out[1]: <xarray.Dataset> Dimensions: (yx: 6) Coordinates: * yx (yx) MultiIndex - y (yx) object 'a' 'a' 'a' 'b' 'b' 'b' # <--- this is index-level - x (yx) int64 1 2 3 1 2 3 # <--- this is also index-level Data variables: foo (yx) int64 1 2 3 4 5 6

In [2]: # 1. indexing a scalar converts index-level x to scalar-level. ...: ds.sel(x=1) Out[2]: <xarray.Dataset> Dimensions: (yx: 2) Coordinates: * yx (yx) MultiIndex - y (yx) object 'a' 'b' # <--- this is index-level - x int64 1 # <--- this is scalar-level Data variables: foo (yx) int64 1 4

In [3]: # 2. indexing a single element from MultiIndex makes a MultiIndex-scalar ...: ds.isel(yx=0) Out[3]: <xarray.Dataset> Dimensions: () Coordinates: yx MultiIndex # <--- this is MultiIndex-scalar - y <U1 'a' - x int64 1 Data variables: foo int64 1

In [6]: # 3. Enables to selecting along a index-level if only one index-level exists in MultiIndex ...: ds.sel(x=1).isel(y=[0,1]) Out[6]: <xarray.Dataset> Dimensions: (yx: 2) Coordinates: * yx (yx) MultiIndex - y (yx) object 'a' 'b' - x int64 1 Data variables: foo (yx) int64 1 4

```

Changes in the public APIs

Some changes were necessary to the public APIs, though I tried to minimize them.

• level_names, get_level_values methods were moved from IndexVariable to Variable. This is because IndexVariable cannnot handle 0-d array, which I want to support in 2.

• scalar_level_names and all_level_names properties were added to Variable

• reset_levels method was added to Variable class to control scalar-level and index-level.

Implementation summary

The main changes in the implementation is the addition of our own wrapper of pd.MultiIndex, PandasMultiIndexAdapter. This does most of MultiIndex-related operations, such as indexing, concatenation, conversion between 'scalar-levelandindex-level`.

What we can do now

The main merit of this proposal is that it enables us to handle MultiIndex more consistent way to the normal Variable. Now we can

• recover the MultiIndex with dropped level. ```python In [5]: ds.sel(x=1).expand_dims('x') Out[5]: <xarray.Dataset> Dimensions: (yx: 2) Coordinates:
• yx (yx) MultiIndex
• y (yx) object 'a' 'b'

• x (yx) int64 1 1 Data variables: foo (yx) int64 1 4 ```

• construct a MultiIndex by concatenation of MultiIndex-scalar. ```python In [8]: xr.concat([ds.isel(yx=i) for i in range(len(ds['yx']))], dim='yx') Out[8]: <xarray.Dataset> Dimensions: (yx: 6) Coordinates:

• yx (yx) MultiIndex
• y (yx) object 'a' 'a' 'a' 'b' 'b' 'b'
• x (yx) int64 1 2 3 1 2 3 Data variables: foo (yx) int64 1 2 3 4 5 6 ```

What we cannot do now

With the current implementation, we can do python ds.sel(y='a').rolling(x=2) but with this PR we cannot, because x is not yet an ordinary coordinate, but a MultiIndex with a single index-level. I think it is better if we can handle such a MultiIndex with a single index-level as very similar way to an ordinary coordinate.

Similary, we can neither do ds.sel(y='a').mean(dim='x'). Also, ds.sel(y='a').to_netcdf('file') (#719)

What are to be decided

• How to repr these new levels (Current formatting is shown in Out[2] and Out[3] above.)
• Terminologies such as index-level, scalar-level, MultiIndex-scalar are clear enough?
• How much operations should we support for a single index-level MultiIndex? Do we support ds.sel(y='a').rolling(x=2) and ds.sel(y='a').mean(dim='x')?

TODOs

• [ ] Support indexing with DataAarray, ds.sel(x=ds.x[0])
• [ ] Support stack, unstack, set_index, reset_index methods with scalar-level MultiIndex.
• [ ] Add a full document
• [ ] Clean up the code related to MultiIndex
• [ ] Fix issues (#1428, #1430, #1431) related to MultiIndex

• [x] Closes #2525
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Started to implement corsen. The API is currently something like python actual = ds.coarsen(time=2, x=3, side='right', coordinate_func={'time': np.max}).max()

Currently, it is not working for a datetime coordinate, since mean does not work for this dtype. e.g. python da = xr.DataArray(np.linspace(0, 365, num=365), dims='time', coords={'time': pd.date_range('15/12/1999', periods=365)}) da['time'].mean() # -> TypeError: ufunc add cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]')

I am not familiar with datetime things. Any advice will be appreciated.

• [x] Closes #2619
• [x] Tests added
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fix using MultiIndex.remove_unused_levels()

• [x] Closes #2468
• [x] Tests added
• [N.A.] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

A fix for #2468.

• [x] Closes #2428

I installed pep8speaks as suggested in #2428. It looks they do not need a yml file, but it may be safer to add this (just renamed from .stickler.yml)

• [x] Closes #2440
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes

It looks that I wrongly remove ddof option for nanstd in #2236. This PR fixes this.

• [x] Closes #2445
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes

It is a regression after #2360.

• [ ] Closes #xxxx
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Started to fix #2363. Now warns a futurewarning in groupby if default reduction dimension is not specified. As a side effect, I added xarray.ALL_DIMS. With dim=ALL_DIMS always reduces along all the dimensions.

• [x] Closes #2230
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

In #2230, the addition of min_count keywords for our reduction methods was discussed, but our duck_array_ops module is becoming messy (mainly due to nan-aggregation methods for dask, bottleneck and numpy) and it looks a little hard to maintain them.

I tried to refactor them by moving nan-aggregation methods to nanops module.

I think I still need to take care of more edge cases, but I appreciate any comment for the current implementation.

Note: In my implementation, bottleneck is not used when skipna=False. bottleneck would be advantageous when skipna=True as numpy needs to copy the entire array once, but I think numpy's method is still OK if skipna=False.

• [x] Closes #1332
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Added xr.gradient, xr.DataArray.gradient, and xr.Dataset.gradient according to #1332.

• [x] Closes #2179
• [x] Tests passed
• [ ] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

• [x] Tests passed (for all non-documentation changes)

Following to #2174

In some methods, consistency of the dictionary arguments and keyword arguments are checked twice in Dataset and Variable. Can we change the API of Variable so that it does not take kwargs-type argument for dimension names?

• [x] Closes #2299
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API.

Now it raises an Error to assign new dimension with the name conflicting with an existing level name. Therefore, it is not allowed ```python b = xr.Dataset(coords={'dim0': ['a', 'b'], 'dim1': [0, 1]}) b = b.stack(dim_stacked=['dim0', 'dim1'])

This should raise an errors even though its length is consistent with b['dim0']

b['c'] = (('dim0',), [10, 11, 12, 13])

This is OK

b['c'] = (('dim_stacked',), [10, 11, 12, 13]) ```

I just mark some to_cdms2 tests xfail. See #2332 for the details. It is a temporal workaround and we may need to keep #2332 open until it is solved.

Dask renamed dask.ghost -> dask.overlap in dask/dask#3830. This PR follows up this.

• [x] Closes #2341
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Now raises a ValueError when the size of input_core_dims is inconsistent with number of argument.

Trivial changes to pass local flake8 tests.

I merged #2312 without making sure the building test passing, but there was a typo. Ths PR fixes it.

• [x] Closes #1831, #1142, #819
• [x] Tests added
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I started to work for refactoring rollings. As suggested in #1831 comment, I implemented rolling_window methods based on as_strided.

I got more than 1,000 times speed up! yey!

python In [1]: import numpy as np ...: import xarray as xr ...: ...: da = xr.DataArray(np.random.randn(10000, 3), dims=['x', 'y']) with the master python %timeit da.rolling(x=5).reduce(np.mean) 1 loop, best of 3: 9.68 s per loop with the current implementation python %timeit da.rolling(x=5).reduce(np.mean) 100 loops, best of 3: 5.29 ms per loop and with the bottleneck python %timeit da.rolling(x=5).mean() 100 loops, best of 3: 2.62 ms per loop

My current concerns are + Can we expose the new rolling_window method of DataArray and Dataset to the public? I think this method itself is useful for many usecases, such as short-term-FFT and convolution. This also gives more flexible rolling operation, such as windowed moving average, strided rolling, and ND-rolling.

• Is there any dask's equivalence to numpy's as_strided? Currently, I just use a slice->concatenate path, but I don't think it is very efficient. (Is it already efficient, as dask utilizes out-of-core computation?)

Any thoughts are welcome.

• [x] Closes #2218
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API.

This adds interp_like, that behaves like reindex_like but using interpolation.

• [x] Closes #2079 (remove if there is no corresponding issue, which should only be the case for minor changes)
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

I started working to add interpolate_at to xarray, as discussed in issue #2079 (but without caching).

I think I need to take care of more edge cases, but before finishing up this PR, I want to discuss what the best API is.

I would like to this method working similar to isel, which may support vectorized interpolation. Currently, this works as follwos

```python In [1]: import numpy as np ...: import xarray as xr ...: ...: da = xr.DataArray([0, 0.1, 0.2, 0.1], dims='x', coords={'x': [0, 1, 2, 3]}) ...:

In [2]: # simple linear interpolation ...: da.interpolate_at(x=[0.5, 1.5]) ...: Out[2]: <xarray.DataArray (x: 2)> array([0.05, 0.15]) Coordinates: * x (x) float64 0.5 1.5

In [3]: # with cubic spline interpolation ...: da.interpolate_at(x=[0.5, 1.5], method='cubic') ...: Out[3]: <xarray.DataArray (x: 2)> array([0.0375, 0.1625]) Coordinates: * x (x) float64 0.5 1.5

In [4]: # interpolation at one single position ...: da.interpolate_at(x=0.5) ...: Out[4]: <xarray.DataArray ()> array(0.05) Coordinates: x float64 0.5

In [5]: # interpolation with broadcasting ...: da.interpolate_at(x=xr.DataArray([[0.5, 1.0], [1.5, 2.0]], dims=['y', 'z'])) ...: Out[5]: <xarray.DataArray (y: 2, z: 2)> array([[0.05, 0.1 ], [0.15, 0.2 ]]) Coordinates: x (y, z) float64 0.5 1.0 1.5 2.0 Dimensions without coordinates: y, z

In [6]: da = xr.DataArray([[0, 0.1, 0.2], [1.0, 1.1, 1.2]], ...: dims=['x', 'y'], ...: coords={'x': [0, 1], 'y': [0, 10, 20]}) ...:

In [7]: # multidimensional interpolation ...: da.interpolate_at(x=[0.5, 1.5], y=[5, 15]) ...: Out[7]: <xarray.DataArray (x: 2, y: 2)> array([[0.55, 0.65], [ nan, nan]]) Coordinates: * x (x) float64 0.5 1.5 * y (y) int64 5 15

In [8]: # multidimensional interpolation with broadcasting ...: da.interpolate_at(x=xr.DataArray([0.5, 1.5], dims='z'), ...: y=xr.DataArray([5, 15], dims='z')) ...: Out[8]: <xarray.DataArray (z: 2)> array([0.55, nan]) Coordinates: x (z) float64 0.5 1.5 y (z) int64 5 15 Dimensions without coordinates: z ```

Design question

1. How many interpolate methods should we support? Currently, I only implemented scipy.interpolate.interp1d for 1dimensional interpolation and scipy.interpolate.RegularGridInterpolator for multidimensional interpolation. I think 90% usecases are linear, but there are more methods in scipy.

2. How do we handle nan? Currently this raises ValueError if nan is present. It may be possible to carry out the interpolation with skipping nan, but in this case the performance would be significantly drops because it cannot be vectorized.

3. Do we support interpolation along dimension without coordinate? In that case, do we attach new coordinate to the object?

4. How should we do if new coordinate has the dimensional coordinate for the dimension to be interpolated? e.g. in the following case, python da = xr.DataArray([0, 0.1, 0.2, 0.1], dims='x', coords={'x': [0, 1, 2, 3]}) rslt = da.interpolate_at(x=xr.DataArray([0.5, 1.5], dims=['x'], coords={'x': [1, 3]}) what would be rslt['x']?

I appreciate any comments.

I noticed an example I added to doc in #2104 consumes more than 1 GB memory, and it results in the failing in readthedocs build.

This PR changes this to a much lighter example.

• [x] Closes #1932
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This precision drop was caused when converting pd.Timestamp to np.array python In [7]: ts = pd.Timestamp(np.datetime64('2018-02-12 06:59:59.999986560')) In [11]: np.asarray(ts, 'datetime64[ns]') Out[11]: array('2018-02-12T06:59:59.999986000', dtype='datetime64[ns]')

We need to call to_datetime64 explicitly.

• [x] Closes #1897
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

I tried to support lazy vectorised indexing inspired by #1897. More tests would be necessary but I want to decide whether it is worth to continue.

My current implementation is + For outer/basic indexers, we combine successive indexers (as we are doing now). + For vectorised indexers, we just store them as is and index sequentially when the evaluation.

The implementation was simpler than I thought, but it has a clear limitation. It requires to load array before the vectorised indexing (I mean, the evaluation time). If we make a vectorised indexing for a large array, the performance significantly drops and it is not noticeable until the evaluation time.

I appreciate any suggestions.

• [x] Related with #2203
• [x] Tests added
• [x] Tests passed
• [x] Fully documented

Related with #2203, I think it is better if xr.DataArray.dot() is working even with older dask, at least in the simpler case (as this is a very primary operation).

The cost is a slight complication of the code. Any comments are welcome.

An example of weighted rolling mean in doc is actually weighted rolling sum. It is a little bit misleading SO, so I propose to change

weighted rolling mean -> weighted rolling sum

• [x] Closes #2112
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API.

Now uses dataset_merge_method when a new coordinate is assigned to a xr.DataArray

• [ ] Closes #N.A.
• [x] Tests added (for all bug fixes or enhancements)
• [ ] Tests N.A.
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I noticed rolling with dask array and with bottleneck can be slightly improved by reducing the padding depth in da.ghost.ghost(a, depth=depth, boundary=boundary).

@jhamman , can you kindly review this?

• [x] Closes #2099
• [x] Tests added
• [x] Tests passed

A quick fix for #2099

• [x] Closes #2113
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Two bugs were found and fixed.

1. rolling a dask-array with center=True and bottleneck
2. rolling an integer dask-array with bottleneck

• [x] Closes #2114
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fixes 2114.

• [x] Closes #2068
• [x] Tests added
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

After this, when assigning a dataarray to a dataset, non-dimensional and conflicted coordinates of the dataarray are dropped.

example ``` In [2]: ds = xr.Dataset({'da': ('x', [0, 1, 2])}, ...: coords={'y': (('x',), [0.1, 0.2, 0.3])}) ...: ds ...: Out[2]: <xarray.Dataset> Dimensions: (x: 3) Coordinates: y (x) float64 0.1 0.2 0.3 Dimensions without coordinates: x Data variables: da (x) int64 0 1 2

In [3]: other = ds['da'] ...: other['y'] = 'x', [0, 1, 2] # conflicted non-dimensional coordinate ...: ds['da'] = other ...: ds ...: Out[3]: <xarray.Dataset> Dimensions: (x: 3) Coordinates: y (x) float64 0.1 0.2 0.3 # 'y' is not overwritten Dimensions without coordinates: x Data variables: da (x) int64 0 1 2 ```

• [x] Closes #1912

The removal of flake8 from travis would increase the clearer separation between style-issue and test failure.

• [x] Closes #1993
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes.

Quick fix of #1993.

With this fix, the script shown in #1993 runs Bottleneck: 0.08317923545837402 s Pandas: 1.3338768482208252 s Xarray: 1.1349339485168457 s

• [x] Closes #1951
• [x] Tests added
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Currently, lazy-einsum for dask is not yet working.

@shoyer I think apply_ufunc supports lazy computation, but I did not yet figure out how to do this. Can you give me a help?

• [x] Partly closes #1944

I noticed this is due to the use of old numpy on rtd.

xref #1956

• [x] Closes #1944

Typo in url.

• [x] Closes #1921
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed (for all non-documentation changes)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Now Variable accepts 0d-np.ndarray indexer.

• [x] Closes #1866
• [x] Tests added (for all bug fixes or enhancements)
• [x] Tests passed
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

I am working to add aggregation ops for object-typed arrays, which may make #1837 cleaner. I added some tests but maybe not sufficient. Any other cases which should be considered? e.g. [True, 3.0, np.nan] etc...

As discussed in #1851, I added a link in doc/installing.rst and a badge on README.

• [x] Relates to #1771

Just added some benchmarks for basic, outer, and vectorized indexing and assignments.

• [x] Closes #1838changes)
• [x] Tests added
• [x] Tests passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fixes #1838. The new rule for reduce is + If dtype is not None and different from array's dtype, use numpy's aggregation function instead of bottleneck's. + If out is not None, raise an error.

as suggested in this comments.

• [x] Closes #1743, #1744
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Found bugs in nputils.NumpyVindexAdapter.__setitem__ and DataArray.__setitem__.

I will add more tests later. Test case suggestions would be appreciated.

• [x] Closes #1720
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Only a single missing underscore causes this issue :) Added tests.

• [x] Closes #1429
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This bug originates from the complicated structure around the array wrappers and their indexing, i.e. different array wrappers support different indexing types, and moreover, some can store another array wrapper in it.

I made some cleanups. + Now every array wrapper is a subclass of NDArrayIndexable + Every array wrapper should implement its own __getitem__ or just store another NDArrayIndexable.

I think I added enough test for it, but I am not yet fully accustomed with xarray's backend. There might be many combinations of their hierarchical relation.

I would appreciate any comments.

• [x] Closes #1694
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This PR simply adds a particular test pointed out in #1694 .

• [x] Closes #1688, #1694
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fixes #1688. It is caused that Variable._broadcast_indexes returns a wrong type of Indexer. Now it supports the orthogonal-indexing with LazilyIndexedArray.

• [x] Closes #1628
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Support #1628.

• [x] Closes #1512
• [x] Tests passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Explicitly deprecated old pandas (< 0.18) and old numpy (< 1.11) supports. Some backported functions in npcompat are removed because numpy == 1.11 already has them.

• [x] Closes #1562
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fixes #1562

• [x] Closes #1593
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [n.a.] Fully documented, including whats-new.rst for all changes and api.rst for new API

Currently some tests fail due to dask bug in #1591

• [x] Closes #1688
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Fixes #1688. It is caused that Variable._broadcast_indexes returns a wrong type of Indexer. Now it supports the orthogonal-indexing with LazilyIndexedArray.

• [x] Closes #1675
• [x] Tests added / passed
• [x] Passes git diff upstream/master **/*py | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Remved T from xr.Dataset.__dir__ to suppress a deprecation warning in Ipython autocompletion.

• [x] Closes #1444, #1436
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

This is a duplicate of #1473 originally opened by @shoyer

Thanks, @shoyer, for giving me github's credit.

I enjoyed this PR. I really appreciate your help to finish up this PR.

• [x] closes #1125
• [x] tests added / passed
• [x] passes git diff upstream/master | flake8 --diff
• [x] whatsnew

Added 1 line to fix #1125.

I hope this is enough. If another care is necessary, please let me know.

• [x] closes #859
• [x] tests added / passed
• [x] passes git diff upstream/master | flake8 --diff
• [x] whatsnew entry

There seems to be two approaches to realize Dataset.rolling, 1. Apply rolling in each DataArrays and then combine them. 2. Apply Dataset directoly with some DataArrays that do not depend on dim kept aside, then merge them later.

I chose the latter approach to reuse existing Rolling object as much as possible, but it results in some duplicates in ImplementsRollingDatasetReduce. Any feedbacks and comments are very welcome.

• [x] closes #1337
• [x] tests added / passed
• [x] passes git diff upstream/master | flake8 --diff
• [x] whatsnew entry

A small fix for the bug reported in #1337, where unselected coords were wrongly assigned as data_vars by sel_points. I hope I did not forget anything.

• [x] closes #1363

Fixes typos in reshaping.rst. Is there a good way to check docs before merge?

• [x] Closes #1408
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

To fix #1408, This modification works, but actually I do not fully satisfied yet. There are if statements in many places.

The major changes I made are 1. variable.__getitem__ now returns an OrderedDict if a single element is selected from MultiIndex. 2. indexing.remap_level_indexers also returns selected_dims which is a map from the original dimension to the selected dims which will be a scalar coordinate.

Change 1 keeps level-coordinates even after ds.isel(yx=0). Change 2 enables to track which levels are selected, then the selected levels are changed to a scalar coordinate.

I guess much smarter solution should exist. I would be happy if anyone gives me a comment.

• [x] closes #1326
• [x] tests added / passed
• [x] passes git diff upstream/master | flake8 --diff
• [x] whatsnew entry

I added a DataArray's method expand_dims based on the discussion in #1326 . The proposed API is similar to numpy.expand_dims and slightly different from Variables.expand_dims, which requires whole sequences of dims of the result array.

My concern is that I do not yet fully understand the lazy data manipulation in xarray. Does Variable.expand_dims do it?