issue_comments

Table actions

4 rows where issue = 374279704 and user = 102827 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • cchwala · 4 ✖

issue 1

  • interpolate_na with limit argument changes size of chunks · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
433454137 https://github.com/pydata/xarray/issues/2514#issuecomment-433454137 https://api.github.com/repos/pydata/xarray/issues/2514 MDEyOklzc3VlQ29tbWVudDQzMzQ1NDEzNw== cchwala 102827 2018-10-26T15:49:20Z 2018-10-31T21:14:48Z CONTRIBUTOR

EDIT: The issue of this post is now separated #2531

I think I have a fix, but wanted to write some failing tests before committing the changes. Doing this I discovered that also DataArray.rolling() does not preserve the chunksizes, apparently depending on the applied method.

```python import pandas as pd import numpy as np import xarray as xr

t = pd.date_range(start='2018-01-01', end='2018-02-01', freq='H') bar = np.sin(np.arange(len(t))) baz = np.cos(np.arange(len(t)))

da_test = xr.DataArray(data=np.stack([bar, baz]), coords={'time': t, 'sensor': ['one', 'two']}, dims=('sensor', 'time'))

print(da_test.chunk({'time': 100}).rolling(time=60).mean().chunks)

print(da_test.chunk({'time': 100}).rolling(time=60).count().chunks) Output for mean: ((2,), (745,)) Output for count: ((2,), (100, 100, 100, 100, 100, 100, 100, 45)) Desired Output: ((2,), (100, 100, 100, 100, 100, 100, 100, 45)) ```

My fix solves my initial problem, but maybe if done correctly it should also solve this bug, too.

Any idea why this depends on whether .mean() or .count() is used?

I have already pushed some WIP changes. Should I already open a PR if though most new test still fail?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   interpolate_na with limit argument changes size of chunks 374279704
433992180 https://github.com/pydata/xarray/issues/2514#issuecomment-433992180 https://api.github.com/repos/pydata/xarray/issues/2514 MDEyOklzc3VlQ29tbWVudDQzMzk5MjE4MA== cchwala 102827 2018-10-29T17:01:12Z 2018-10-29T17:01:12Z CONTRIBUTOR

@dcherian Okay. A WIP PR will follow, but might take some days.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   interpolate_na with limit argument changes size of chunks 374279704
433369567 https://github.com/pydata/xarray/issues/2514#issuecomment-433369567 https://api.github.com/repos/pydata/xarray/issues/2514 MDEyOklzc3VlQ29tbWVudDQzMzM2OTU2Nw== cchwala 102827 2018-10-26T10:53:32Z 2018-10-26T10:53:32Z CONTRIBUTOR

Thanks @fujiisoup for the quick response and the pointers. I will have a look and report back if a PR is within my capabilities or not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   interpolate_na with limit argument changes size of chunks 374279704
433346685 https://github.com/pydata/xarray/issues/2514#issuecomment-433346685 https://api.github.com/repos/pydata/xarray/issues/2514 MDEyOklzc3VlQ29tbWVudDQzMzM0NjY4NQ== cchwala 102827 2018-10-26T09:27:19Z 2018-10-26T09:27:19Z CONTRIBUTOR

The problem seems to occur here

https://github.com/pydata/xarray/blob/5940100761478604080523ebb1291ecff90e779e/xarray/core/missing.py#L368-L376

because of the usage of .construct(). A quick try without it, shows that the chunksize is preserved then.

Hence, .construct() might need a fix for correctly dealing with the chunks of dask.arrays.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   interpolate_na with limit argument changes size of chunks 374279704

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);