issues

What is your issue?

Currently, the 'limit' argument of interpolate_na shows some counterintuitive/undocumented behaviour. Take the following example: python import xarray as xr import numpy as np n=np.nan da=xr.DataArray([n, n, n, 4, 5, n ,n ,n], dims=["y"]) da.interpolate_na('y', limit=1, fill_value='extrapolate') This will produce the following result: array([ 1., nan, nan, 4., 5., 6., nan, nan]) Two things are surprising, in my opinion:

1. The interpolated value 1 at the beginning is far from any of the given values
2. The filling is done only towards the 'right'. This asymmetric behaviour is not mentioned in the documentation.

Comparison to pandas

Similar behaviour can be created using pandas with the following arguments: python da=xr.DataArray([n, n, n, 4, 5, n ,n ,n], dims=["y"]) dap=da.to_pandas() dap.interpolate(method='slinear', limit=1, limit_direction='forward', fill_value='extrapolate')

This is equivalent to the current xarray behaviour, except there is no 1 at the beginning.

Cause

Currently, the fill mask in xarray is implemented using a rolling window operation, where values outside the array are assumed to be valid (therefore the 1). See xarray.core.missing._get_valid_fill_mask

Possible Solutions

Boundary Issue

Concerning the 1 at the beginning: I think this should be considered a bug. It is likely not what you would expect if you specify a limit. As stated, pandas does not create it as well.

Asymmetric Filling

Concerning the asymmetric filling, I see two options: 1. No changes to the code, but mention in the documentation that (effectively), a forward-fill is done. 2. Make something similar to what pandas is doing. In pandas, there are two additional arguments controlling the limit behaviour: limit_direction is controlling the fill direction (left, right or both). limit_area effectively controls if we only do interpolation or allow for extrapolation as well.

What do you think?

What is your issue?

If an array contains np.inf and a rolling operation is applied, all values after this one are nan if numbagg is used. Take the following example:

python import xarray as xr import numpy as np xr.set_options(use_numbagg=False) da=xr.DataArray([1,2,3,np.inf,4,5,6,7,8,9,10], dims=['x']) da.rolling(x=2).sum() Output <xarray.DataArray (x: 11)> Size: 88B array([nan, 3., 5., inf, inf, 9., 11., 13., 15., 17., 19.]) Dimensions without coordinates: x With Numbagg: python xr.set_options(use_numbagg=True) da=xr.DataArray([1,2,3,np.inf,4,5,6,7,8,9,10], dims=['x']) print(da.rolling(x=2).sum()) Output <xarray.DataArray (x: 11)> Size: 88B array([nan, 3., 5., inf, inf, nan, nan, nan, nan, nan, nan]) Dimensions without coordinates: x

What did I expect?

I expected no user-visible changes in the output values if numbagg is activated.

Maybe, this is not a bug, but expected behaviour for numbagg. The following warning was raised from the second call: .../Local/virtual_environments/xarray_performance/lib/python3.10/site-packages/numbagg/decorators.py:247: RuntimeWarning: invalid value encountered in move_sum return gufunc(*arr, window, min_count, axis=axis, **kwargs)

If this is expected, I think it would be good to have a page in the documentation which lists the downsides and limitations of the various tool to accelerate xarray. From the current installation docs, I assumed I just need to install numbagg/bottleneck to make xarray faster without any changes in output values.

Environment

xarray==2024.2.0 numbagg==0.8.0

What happened?

In the case of multidimensional arrays, the max_gap argument of interpolate_na is currently not working correctly at the array boundaries. This is likely due to a missing "dim" argument in the max() aggregation in xarray.core.missing._get_nan_block_lengths, I think.

What did you expect to happen?

In the following code example, due to max_gap=2, no extrapolation should be performed for the second row. Currently, this is the case, the output created is:

<xarray.DataArray (x: 2, y: 5)> array([[1., 2., 3., 4., 5.], [1., 2., 3., 4., 5.]]) Coordinates: * x (x) int64 0 1 * y (y) int64 0 1 2 3 4

Minimal Complete Verifiable Example

Python import xarray as xr import numpy as np da=xr.DataArray([[1, 2,3,4, np.nan],[1,2, np.nan, np.nan, np.nan]], coords=[('x', [0,1]), ('y', [0,1,2,3,4])]) da_interp=da.interpolate_na(dim='y', max_gap=2, fill_value='extrapolate') print(da_interp)

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

I added the missing dim argument and adapted the test cases (Currently, there was no test case for fully multidimensional arrays with a gap at the end).

Environment