home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 456988170

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2699#issuecomment-456988170 https://api.github.com/repos/pydata/xarray/issues/2699 456988170 MDEyOklzc3VlQ29tbWVudDQ1Njk4ODE3MA== 1217238 2019-01-23T22:17:21Z 2019-01-23T22:17:38Z MEMBER

Thanks for the clear report. Indeed, this looks like a bug.

bfill() and ffill() are implemented on dask arrays via apply_ufunc, but they're applied independently on each chunk -- there's no filling between chunks: https://github.com/pydata/xarray/blob/ddacf405fb256714ce01e1c4c464f829e1cc5058/xarray/core/missing.py#L262-L289

Instead, I think we need a multi-step process for parallelizing bottleneck.push, e.g., 1. Forward fill each chunk independently. 2. Slice out the last element of each chunk and forward fill these. 3. Prepend filled last elements to the start of each chunk, and forward fill them again.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  402413097
Powered by Datasette · Queries took 0.649ms · About: xarray-datasette