issues

Table actions

1 row where repo = 13221727, type = "issue" and user = 19997801 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 1

  • issue · 1 ✖

state 1

repo 1

  • xarray · 1 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1396969695 I_kwDOAMm_X85TRBDf 7128 `rolling().sum()` is numerically unstable ilemhadri 19997801 closed 0     4 2022-10-04T23:39:31Z 2022-11-30T01:43:26Z 2022-11-30T01:43:26Z NONE      

What happened?

On an input array like mydata.values[0,0,:] array([0. , 0. , 0. , 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0. , 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 0.57392103, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 2.29551022, 0.57383408, 0.57383408, 0.57383408, 0.57383408, 0.57383408, 0.57383408, 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ])

that has all positive values, and then zeros, computing the rolling sum (or mean) leads to negative values mydata.rolling(t=3, min_periods=1).sum().values[0,0,:] array([-3.33066907e-14, -3.33066907e-14, -3.33066907e-14, 5.73921029e-01, 2.14784206e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.14784206e+00, 1.14784206e+00, 1.14784206e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 1.72176309e+00, 3.44335228e+00, 5.16494146e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 6.88653065e+00, 5.16485452e+00, 3.44317838e+00, 1.72150224e+00, 1.72150224e+00, 1.72150224e+00, 1.72150224e+00, 1.14766816e+00, 5.73834081e-01, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14, -3.35287353e-14])

Both arrays have dtype = float64.

The issue aggravates as the rolling window increases.

What did you expect to happen?

the rolling calculation could be more numerically precise by keeping track for instance of the Kahan compensation term. https://en.wikipedia.org/wiki/Kahan_summation_algorithm

MVCE confirmation

  • [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • [X] Complete example — the example is self-contained, including all data and the text of any traceback.
  • [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • [X ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Environment

this is reproducible across xarray versions, but mine is 2022.09.0.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7128/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);