home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 1637898633

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1637898633 I_kwDOAMm_X85hoFmJ 7665 Interpolate_na: Rework 'limit' argument documentation/implementation 42680748 open 0     6 2023-03-23T16:46:39Z 2024-03-13T17:53:58Z   CONTRIBUTOR      

What is your issue?

Currently, the 'limit' argument of interpolate_na shows some counterintuitive/undocumented behaviour. Take the following example: python import xarray as xr import numpy as np n=np.nan da=xr.DataArray([n, n, n, 4, 5, n ,n ,n], dims=["y"]) da.interpolate_na('y', limit=1, fill_value='extrapolate') This will produce the following result: array([ 1., nan, nan, 4., 5., 6., nan, nan]) Two things are surprising, in my opinion:

  1. The interpolated value 1 at the beginning is far from any of the given values
  2. The filling is done only towards the 'right'. This asymmetric behaviour is not mentioned in the documentation.

Comparison to pandas

Similar behaviour can be created using pandas with the following arguments: python da=xr.DataArray([n, n, n, 4, 5, n ,n ,n], dims=["y"]) dap=da.to_pandas() dap.interpolate(method='slinear', limit=1, limit_direction='forward', fill_value='extrapolate')

Output ``` y 0 NaN 1 NaN 2 NaN 3 4.0 4 5.0 5 6.0 6 NaN 7 NaN dtype: float64 ```

This is equivalent to the current xarray behaviour, except there is no 1 at the beginning.

Cause

Currently, the fill mask in xarray is implemented using a rolling window operation, where values outside the array are assumed to be valid (therefore the 1). See xarray.core.missing._get_valid_fill_mask

Possible Solutions

Boundary Issue

Concerning the 1 at the beginning: I think this should be considered a bug. It is likely not what you would expect if you specify a limit. As stated, pandas does not create it as well.

Asymmetric Filling

Concerning the asymmetric filling, I see two options: 1. No changes to the code, but mention in the documentation that (effectively), a forward-fill is done. 2. Make something similar to what pandas is doing. In pandas, there are two additional arguments controlling the limit behaviour: limit_direction is controlling the fill direction (left, right or both). limit_area effectively controls if we only do interpolation or allow for extrapolation as well.

What do you think?

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7665/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 2 rows from issue in issue_comments
Powered by Datasette · Queries took 1.093ms · About: xarray-datasette