{"database": "github", "table": "issue_comments", "is_view": false, "human_description_en": "where issue = 402413097 and user = 1217238 sorted by updated_at descending", "rows": [["https://github.com/pydata/xarray/issues/2699#issuecomment-456999707", "https://api.github.com/repos/pydata/xarray/issues/2699", 456999707, "MDEyOklzc3VlQ29tbWVudDQ1Njk5OTcwNw==", 1217238, "2019-01-23T22:58:50Z", "2019-01-23T23:13:59Z", "MEMBER", "I think this will work (though it needs more tests):\r\n```python\r\nimport bottleneck\r\nimport dask.array as da\r\nimport numpy as np\r\n\r\ndef _last_element(array, axis):\r\n  slices = [slice(None)] * array.ndim\r\n  slices[axis] = slice(-1, None)\r\n  return array[tuple(slices)]\r\n\r\ndef _concat_push_slice(last_elements, array, axis):\r\n  concatenated = np.concatenate([last_elements, array], axis=axis)\r\n  pushed = bottleneck.push(concatenated, axis=axis)\r\n  slices = [slice(None)] * array.ndim\r\n  slices[axis] = slice(1, None)\r\n  sliced = pushed[tuple(slices)]\r\n  return sliced\r\n\r\ndef push(array, axis):\r\n  if axis < 0:\r\n    axis += array.ndim\r\n  pushed = array.map_blocks(bottleneck.push, dtype=array.dtype, axis=axis)\r\n  new_chunks = list(array.chunks)\r\n  new_chunks[axis] = tuple(1 for _ in array.chunks[axis])\r\n  last_elements = pushed.map_blocks(\r\n      _last_element, dtype=array.dtype, chunks=tuple(new_chunks), axis=axis)\r\n  pushed_last_elements = (\r\n      last_elements.rechunk({axis: -1})\r\n      .map_blocks(bottleneck.push, dtype=array.dtype, axis=axis)\r\n      .rechunk({axis: 1})\r\n  )\r\n  nan_shape = tuple(1 if axis == a else s for a, s in enumerate(array.shape))\r\n  nan_chunks = tuple((1,) if axis == a else c for a, c in enumerate(array.chunks))\r\n  shifted_pushed_last_elements = da.concatenate(\r\n      [da.full(np.nan, shape=nan_shape, chunks=nan_chunks),\r\n       pushed_last_elements[(slice(None),) * axis + (slice(None, -1),)]],\r\n      axis=axis)\r\n  return da.map_blocks(\r\n      _concat_push_slice,\r\n      shifted_pushed_last_elements,\r\n      pushed,\r\n      dtype=array.dtype,\r\n      chunks=array.chunks,\r\n      axis=axis,\r\n  )\r\n\r\n# tests\r\narray = np.array([np.nan, np.nan, np.nan, 1, 2, 3,\r\n                  np.nan, np.nan, 4, 5, np.nan, 6])\r\nexpected = bottleneck.push(array, axis=0)\r\nfor c in range(1, 11):\r\n  actual = push(da.from_array(array, chunks=c), axis=0).compute()\r\n  np.testing.assert_equal(actual, expected)\r\n```", "{\"total_count\": 3, \"+1\": 3, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", null, 402413097], ["https://github.com/pydata/xarray/issues/2699#issuecomment-456988170", "https://api.github.com/repos/pydata/xarray/issues/2699", 456988170, "MDEyOklzc3VlQ29tbWVudDQ1Njk4ODE3MA==", 1217238, "2019-01-23T22:17:21Z", "2019-01-23T22:17:38Z", "MEMBER", "Thanks for the clear report. Indeed, this looks like a bug.\r\n\r\n`bfill()` and `ffill()` are implemented on dask arrays via `apply_ufunc`, but they're applied independently on each chunk -- there's no filling between chunks:\r\nhttps://github.com/pydata/xarray/blob/ddacf405fb256714ce01e1c4c464f829e1cc5058/xarray/core/missing.py#L262-L289\r\n\r\nInstead, I think we need a multi-step process for parallelizing `bottleneck.push`, e.g.,\r\n1. Forward fill each chunk independently.\r\n2. Slice out the *last element* of each chunk and forward fill these.\r\n3. Prepend filled last elements to the start of each chunk, and forward fill them again.", "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", null, 402413097]], "truncated": false, "filtered_table_rows_count": 2, "expanded_columns": [], "expandable_columns": [[{"column": "issue", "other_table": "issues", "other_column": "id"}, "title"], [{"column": "user", "other_table": "users", "other_column": "id"}, "login"]], "columns": ["html_url", "issue_url", "id", "node_id", "user", "created_at", "updated_at", "author_association", "body", "reactions", "performed_via_github_app", "issue"], "primary_keys": ["id"], "units": {}, "query": {"sql": "select html_url, issue_url, id, node_id, user, created_at, updated_at, author_association, body, reactions, performed_via_github_app, issue from issue_comments where \"issue\" = :p0 and \"user\" = :p1 order by updated_at desc limit 101", "params": {"p0": "402413097", "p1": "1217238"}}, "facet_results": {"user": {"name": "user", "type": "column", "hideable": false, "toggle_url": "/github/issue_comments.json?issue=402413097&user=1217238", "results": [{"value": 1217238, "label": "shoyer", "count": 2, "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?issue=402413097", "selected": true}], "truncated": false}}, "suggested_facets": [], "next": null, "next_url": null, "private": false, "allow_execute_sql": true, "query_ms": 8467.713524005376}