issue_comments

Table actions

5 rows where author_association = "MEMBER" and issue = 269297904 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

issue 1

  • Could a DataArrayRolling object compute an arbitrary function on rolling windows? · 5 ✖

author_association 1

  • MEMBER · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
438357503 https://github.com/pydata/xarray/issues/1666#issuecomment-438357503 https://api.github.com/repos/pydata/xarray/issues/1666 MDEyOklzc3VlQ29tbWVudDQzODM1NzUwMw== fujiisoup 6815844 2018-11-13T17:21:38Z 2018-11-13T17:21:38Z MEMBER

E.g. consider a 1000x1000x1000 array for which I want to apply rolling window of length 500 along the final dimension; I believe Rolling.construct/reduce will construct a 1000x1000x1000x500 array.

construct method does not allocate that large array in memory. It uses the strided trick and therefore consumers only the order of 1000x1000x1000. But sometimes the user function does not like strangely strided array, e.g. nansum allocate large array with 1000x1000x1000x500 internally.

What function are you using?

Maybe we can add apply method but I think we need to decide a good balance between memory and computational efficiency.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Could a DataArrayRolling object compute an arbitrary function on rolling windows? 269297904
438324506 https://github.com/pydata/xarray/issues/1666#issuecomment-438324506 https://api.github.com/repos/pydata/xarray/issues/1666 MDEyOklzc3VlQ29tbWVudDQzODMyNDUwNg== fujiisoup 6815844 2018-11-13T16:12:25Z 2018-11-13T16:12:25Z MEMBER

Maybe we need a better API similar to Groupby.apply, but it can be doable by construct method that gives another DataArray with windowed dimension attached to the last position.

We can apply any function on this constructed DataArray.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Could a DataArrayRolling object compute an arbitrary function on rolling windows? 269297904
340923959 https://github.com/pydata/xarray/issues/1666#issuecomment-340923959 https://api.github.com/repos/pydata/xarray/issues/1666 MDEyOklzc3VlQ29tbWVudDM0MDkyMzk1OQ== jhamman 2443309 2017-10-31T22:16:07Z 2017-10-31T22:16:07Z MEMBER

@ziofil - We don't support indexing on the rolling iterator at the moment. You could just grab the windows you want in this way:

```Python w1 = 2 w2 = 7 for i, (k, v) in enumerate(r): if i > w1 and i <= w2: print(k.item(), v.values)

3 [2 3] 4 [3 4] 5 [4 5] 6 [5 6] 7 [6 7] ```

By the way, I think we'd be happy to see some feature development on this front if you are interested in developing the rolling objects out further.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Could a DataArrayRolling object compute an arbitrary function on rolling windows? 269297904
340241366 https://github.com/pydata/xarray/issues/1666#issuecomment-340241366 https://api.github.com/repos/pydata/xarray/issues/1666 MDEyOklzc3VlQ29tbWVudDM0MDI0MTM2Ng== jhamman 2443309 2017-10-29T06:46:01Z 2017-10-29T06:46:01Z MEMBER

@shoyer - this is really odd. Its probably coming from https://github.com/pydata/xarray/blob/master/xarray/core/rolling.py#L149-L150 but I don't remember exactly what/why we're trying to accomplish here.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Could a DataArrayRolling object compute an arbitrary function on rolling windows? 269297904
340214704 https://github.com/pydata/xarray/issues/1666#issuecomment-340214704 https://api.github.com/repos/pydata/xarray/issues/1666 MDEyOklzc3VlQ29tbWVudDM0MDIxNDcwNA== shoyer 1217238 2017-10-28T19:36:07Z 2017-10-28T19:36:07Z MEMBER

You can iterate through a rolling object as a generator, e.g., In [6]: for k, v in r: ...: print(k.item(), v.values) ...: 0 [ True] 1 [0 1] 2 [1 2] 3 [2 3] 4 [3 4] 5 [4 5] 6 [5 6] 7 [6 7] 8 [7 8] 9 [8 9] Does this help?

Note that something weird seems to be going on with the first time window. I don't know why it's a boolean. @jhamman any idea what's going on?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Could a DataArrayRolling object compute an arbitrary function on rolling windows? 269297904

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);