issues

Code Sample, a copy-pastable example if possible

```python import pandas as pd import xarray as xr import numpy as np

t = pd.date_range(start='2018-01-01', end='2018-02-01', freq='H') foo = np.sin(np.arange(len(t))) bar = np.cos(np.arange(len(t)))

foo[1] = np.NaN bar[2] = np.NaN

ds_test = xr.Dataset(data_vars={'foo': ('time', foo), 'bar': ('time', bar)}, coords={'time': t}).chunk()

print(ds_test) print("\n\n### After .interpolate_na(dim='time')\n") print(ds_test.interpolate_na(dim='time')) print("\n\n### After .interpolate_na(dim='time', limit=5)\n") print(ds_test.interpolate_na(dim='time', limit=5)) print("\n\n### After .interpolate_na(dim='time', limit=20)\n") print(ds_test.interpolate_na(dim='time', limit=20)) ```

Output of the above code. Note the different chunk sizes, depending on the value of limit: ``` <xarray.Dataset> Dimensions: (time: 745) Coordinates: * time (time) datetime64[ns] 2018-01-01 2018-01-01T01:00:00 ... 2018-02-01 Data variables: foo (time) float64 dask.array<shape=(745,), chunksize=(745,)> bar (time) float64 dask.array<shape=(745,), chunksize=(745,)>

After .interpolate_na(dim='time')

<xarray.Dataset> Dimensions: (time: 745) Coordinates: * time (time) datetime64[ns] 2018-01-01 2018-01-01T01:00:00 ... 2018-02-01 Data variables: foo (time) float64 dask.array<shape=(745,), chunksize=(745,)> bar (time) float64 dask.array<shape=(745,), chunksize=(745,)>

After .interpolate_na(dim='time', limit=5)

<xarray.Dataset> Dimensions: (time: 745) Coordinates: * time (time) datetime64[ns] 2018-01-01 2018-01-01T01:00:00 ... 2018-02-01 Data variables: foo (time) float64 dask.array<shape=(745,), chunksize=(3,)> bar (time) float64 dask.array<shape=(745,), chunksize=(3,)>

After .interpolate_na(dim='time', limit=20)

<xarray.Dataset> Dimensions: (time: 745) Coordinates: * time (time) datetime64[ns] 2018-01-01 2018-01-01T01:00:00 ... 2018-02-01 Data variables: foo (time) float64 dask.array<shape=(745,), chunksize=(10,)> bar (time) float64 dask.array<shape=(745,), chunksize=(10,)> ```

Problem description

When using xarray.DataArray.interpolate_na() with the limit kwarg this changes the chunksize of the resulting dask.arrays.

Expected Output

The chunksize should not change. Very small chunks which results from typical small values of limit are not optimal for the performance of dask. Also, things like .rolling() will fail if the chunksize is smaller than the window length of the rolling window.

Output of xr.show_versions()

• [ ] Closes #2514
• [ ] Closes #2531
• [ ] Tests added (for all bug fixes or enhancements)
• [ ] Fully documented, including whats-new.rst for all changes

Short summary

The two rolling-window functions for dask.array * dask_rolling_wrapper * rolling_window

will be fixed to preserve dask.array chunksizes.

Long summary

The specific initial problem with chunksizes and interpolate_na() in #2514 is caused by the padding done in

https://github.com/pydata/xarray/blob/5940100761478604080523ebb1291ecff90e779e/xarray/core/dask_array_ops.py#L74-L85

which adds a small array with a small chunk to the initial array.

There is another related problem where DataArray.rolling() changes the size and distribution of dask.array chunks which stems from this code

https://github.com/pydata/xarray/blob/b622c5e7da928524ef949d9e389f6c7f38644494/xarray/core/dask_array_ops.py#L23

For some (historic) reason there are these two rolling-window functions for dask. Both need to be fixed to preserve chunksize of a dask.array in all cases.

This issue was found and discussed in the related issue #2514

I open a separate issue for clarity.

Code Sample, a copy-pastable example if possible

```python import pandas as pd import numpy as np import xarray as xr

t = pd.date_range(start='2018-01-01', end='2018-02-01', freq='H') bar = np.sin(np.arange(len(t))) baz = np.cos(np.arange(len(t)))

da_test = xr.DataArray(data=np.stack([bar, baz]), coords={'time': t, 'sensor': ['one', 'two']}, dims=('sensor', 'time'))

print(da_test.chunk({'time': 100}).rolling(time=60).mean().chunks)

print(da_test.chunk({'time': 100}).rolling(time=60).count().chunks) Output for mean: ((2,), (745,)) Output for count: ((2,), (100, 100, 100, 100, 100, 100, 100, 45)) Desired Output: ((2,), (100, 100, 100, 100, 100, 100, 100, 45)) ```

Problem description

DataArray.rolling() does not preserve the chunksizes, apparently depending on the applied method.

Output of xr.show_versions()

• [x] Closes #1399
• [x] Tests added / passed
• [x] Passes git diff upstream/master | flake8 --diff
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Instead of casting the input numeric dates to float, they are now casted to nanoseconds as int64 which makes pd.to_timedelta() work much faster (x100 speedup on my machine).

On my machine all existing tests for conventions.py pass. Overflows should be handled by these two already existing lines since everything in the valid range of pd.to_datetime should be save.

Hi, decode_cf_datetime is slowed down because it always passes floats to pd.to_timedelta, while pd.to_timedelta is much faster when working on integers.

Here is a notebook that shows the differences. Working with integers is approx. one order of magnitude faster.

Hence, it would be great to automatically do the conversion from raw time value floats to integers in nanoseconds where possible (likely limited to resolutions bellow days or hours to avoid coping with different durations numbers of nanoseconds within e.g. different months).

As alternative, maybe avoid forcing the cast to floats and indicate in the docstring that the raw values should be integers to speed up the conversion.

This could possibly also be resolved in pd.to_timedelta but I assume it will be more complicated to deal with all the edge cases there.