html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1666#issuecomment-708888589,https://api.github.com/repos/pydata/xarray/issues/1666,708888589,MDEyOklzc3VlQ29tbWVudDcwODg4ODU4OQ==,26384082,2020-10-15T04:25:27Z,2020-10-15T04:25:27Z,NONE,"In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity
If this issue remains relevant, please comment here or remove the `stale` label; otherwise it will be marked as closed automatically
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438749995,https://api.github.com/repos/pydata/xarray/issues/1666,438749995,MDEyOklzc3VlQ29tbWVudDQzODc0OTk5NQ==,7441788,2018-11-14T17:35:17Z,2018-11-14T17:38:31Z,CONTRIBUTOR,"Also, the following code seems to accomplish the same as the above:
```
def apply_func_rolling(func, *args, **kwargs):
# determine rolling parameters, and remove them from kwargs
apply_func_kwargs = {'input_core_dims', 'output_core_dims', 'vectorize', 'join', 'dataset_join',
'keep_attrs', 'exclude_dims', 'dataset_fill_value', 'kwargs', 'dask',
'output_dtypes', 'output_sizes'}
min_periods = kwargs.pop('min_periods', None)
center = kwargs.pop('center', False)
dim = xr.core.utils.either_dict_or_kwargs(kwargs.pop('dim', None),
{k: v for k, v in kwargs.items()
if k not in apply_func_kwargs},
'apply_func_rolling')
if len(dim) != 1:
raise ValueError(""precisely one rolling dimension must be specified"")
rolling_dim = list(dim.keys())[0]
kwargs.pop(rolling_dim)
temp_rolling_dim = '__temp__{}__'.format(rolling_dim)
# change input_core_dims rolling_dim values to temp_rolling_dim
input_core_dims = kwargs.get('input_core_dims', None)
if input_core_dims:
kwargs['input_core_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_)
for dim_ in dims_] for dims_ in input_core_dims]
# change exclude_dims rolling_dim values to temp_rolling_dim
exclude_dims = kwargs.get('exclude_dims', None)
if exclude_dims:
kwargs['exclude_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_)
for dim_ in dims_] for dims_ in exclude_dims]
# call apply_func() with rolling-constructed objects
return xr.apply_ufunc(func,
*[(arg.rolling(dim=dim, min_periods=min_periods, center=center).
construct(temp_rolling_dim) if (rolling_dim in arg.dims) else arg)
for arg in args],
**kwargs)
apply_func_rolling(lambda a, b, w: ...,
variables, observations, weights,
date=N,
input_core_dims=[['date', 'dim1', 'dim2', 'var'],
['date', 'dim1', 'dim2'],
['date', 'dim1', 'dim2']],
output_core_dims=[['var']],
vectorize=True)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438372589,https://api.github.com/repos/pydata/xarray/issues/1666,438372589,MDEyOklzc3VlQ29tbWVudDQzODM3MjU4OQ==,7441788,2018-11-13T17:56:17Z,2018-11-13T20:16:57Z,CONTRIBUTOR,"> construct method does not allocate that large array in memory. It uses the strided trick and therefore consumers only the order of 1000x1000x1000.
Ah. I didn't realize that. Good to know.
What I'm actually looking to do is a rolling weighted regression. I have three DataArrays:
- observations, dims=('date', 'dim1', 'dim2')
- variables, dims=('date', 'dim1', 'dim2', 'var')
- weights, dims=('date', 'dim1', 'dim2')
I want to calculate a regression_coefficients DataArray with dims=('date', 'var'), where for each date it has the weighted regression coefficients calculated over the trailing N dates (over 'dim1' and 'dim2'). One way would be to put the three DataArrays in a Dataset, and then use a newly-defined `Dataset.rolling().apply()`. Another way would be to use an enhanced version of `apply_ufunc()` that can take `Rolling` objects. But now that I know that `DataArrayRolling.construct()` won't kill my machine, I'll try `apply_ufunc()` with the three `DataArrayRolling.construct()` objects. I'd welcome other suggestions.
OK, I seem to have got my problem working using:
```
apply_ufunc(lambda a, b, w: ...,
variables.rolling(date=N).construct('temp_date'),
observations.rolling(date=N).construct('temp_date'),
weights.rolling(date=N).construct('temp_date'),
input_core_dims=[['temp_date', 'dim1', 'dim2', 'var'],
['temp_date', 'dim1', 'dim2'],
['temp_date', 'dim1', 'dim2']],
output_core_dims=[['var']],
vectorize=True)
```
Still, I wonder if there isn't a more ""natural"" way of accomplishing this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438357503,https://api.github.com/repos/pydata/xarray/issues/1666,438357503,MDEyOklzc3VlQ29tbWVudDQzODM1NzUwMw==,6815844,2018-11-13T17:21:38Z,2018-11-13T17:21:38Z,MEMBER,"> E.g. consider a 1000x1000x1000 array for which I want to apply rolling window of length 500 along the final dimension; I believe Rolling.construct/reduce will construct a 1000x1000x1000x500 array.
construct method does not allocate that large array in memory. It uses the strided trick and therefore consumers only the order of 1000x1000x1000.
But sometimes the user function does not like strangely strided array, e.g. nansum allocate large array with 1000x1000x1000x500 internally.
What function are you using?
Maybe we can add apply method but I think we need to decide a good balance between memory and computational efficiency.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438351586,https://api.github.com/repos/pydata/xarray/issues/1666,438351586,MDEyOklzc3VlQ29tbWVudDQzODM1MTU4Ng==,7441788,2018-11-13T17:06:38Z,2018-11-13T17:06:38Z,CONTRIBUTOR,"I think there are actually a couple different ways `Rolling.apply` could work, but this seems like one possible way:
```
from xarray.core.utils import maybe_wrap_array
from xarray.core.combine import concat
def rolling_apply(rolling, func, *args, **kwargs):
applied = [maybe_wrap_array(label, func(arr, *args, **kwargs)) for label, arr in rolling]
combined = concat(applied, dim=rolling.obj.coords[rolling.dim])
return combined
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438345386,https://api.github.com/repos/pydata/xarray/issues/1666,438345386,MDEyOklzc3VlQ29tbWVudDQzODM0NTM4Ng==,7441788,2018-11-13T16:50:08Z,2018-11-13T16:52:13Z,CONTRIBUTOR,The problem I have with `Rolling.construct` is the same that I have with `Rolling.reduce`: it's very (potentially) memory-inefficient. E.g. consider a 1000x1000x1000 array for which I want to apply rolling window of length 500 along the final dimension; I believe `Rolling.construct/reduce` will construct a 1000x1000x1000x500 array. This can quickly get out of hand.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438344608,https://api.github.com/repos/pydata/xarray/issues/1666,438344608,MDEyOklzc3VlQ29tbWVudDQzODM0NDYwOA==,7441788,2018-11-13T16:48:09Z,2018-11-13T16:48:09Z,CONTRIBUTOR,"Separately, maybe `apply_ufunc` should accepts `Rolling` objects?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438324506,https://api.github.com/repos/pydata/xarray/issues/1666,438324506,MDEyOklzc3VlQ29tbWVudDQzODMyNDUwNg==,6815844,2018-11-13T16:12:25Z,2018-11-13T16:12:25Z,MEMBER,"Maybe we need a better API similar to Groupby.apply, but it can be doable by [`construct` method](http://xarray.pydata.org/en/stable/generated/xarray.core.rolling.DataArrayRolling.construct.html#xarray.core.rolling.DataArrayRolling.construct) that gives another DataArray with windowed dimension attached to the last position.
We can apply any function on this constructed DataArray.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438322013,https://api.github.com/repos/pydata/xarray/issues/1666,438322013,MDEyOklzc3VlQ29tbWVudDQzODMyMjAxMw==,7441788,2018-11-13T16:06:24Z,2018-11-13T16:06:24Z,CONTRIBUTOR,I think what is needed are `DataArrayRolling.apply` and `DatasetRolling.apply` (like `DataArrayGroupBy.apply` and `DatasetGroupBy.apply`). The problem with the `reduce` methods is that they are memory-inefficient.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-341760223,https://api.github.com/repos/pydata/xarray/issues/1666,341760223,MDEyOklzc3VlQ29tbWVudDM0MTc2MDIyMw==,8944955,2017-11-03T16:43:42Z,2017-11-03T16:44:40Z,NONE,"I'd love to, I can take a look at the code, but I'm afraid my skills are not up to the task yet...
How would one implement slicing and indexing? By adding a method to DataArrayRolling?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-340923959,https://api.github.com/repos/pydata/xarray/issues/1666,340923959,MDEyOklzc3VlQ29tbWVudDM0MDkyMzk1OQ==,2443309,2017-10-31T22:16:07Z,2017-10-31T22:16:07Z,MEMBER,"@ziofil - We don't support indexing on the rolling iterator at the moment. You could just grab the windows you want in this way:
```Python
w1 = 2
w2 = 7
for i, (k, v) in enumerate(r):
if i > w1 and i <= w2:
print(k.item(), v.values)
3 [2 3]
4 [3 4]
5 [4 5]
6 [5 6]
7 [6 7]
```
By the way, I think we'd be happy to see some feature development on this front if you are interested in developing the rolling objects out further.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-340594218,https://api.github.com/repos/pydata/xarray/issues/1666,340594218,MDEyOklzc3VlQ29tbWVudDM0MDU5NDIxOA==,8944955,2017-10-30T21:44:39Z,2017-10-31T13:42:02Z,NONE,"I didn't know I could just iterate through it! Thank you. It's very fast too.
One thing though: I can iterate through the whole thing, but as it doesn't support indexing and it's not subscriptable, how do I extract the first `n` windows? Or from window `w1` to window `w2`?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-340241366,https://api.github.com/repos/pydata/xarray/issues/1666,340241366,MDEyOklzc3VlQ29tbWVudDM0MDI0MTM2Ng==,2443309,2017-10-29T06:46:01Z,2017-10-29T06:46:01Z,MEMBER,@shoyer - this is really odd. Its probably coming from https://github.com/pydata/xarray/blob/master/xarray/core/rolling.py#L149-L150 but I don't remember exactly what/why we're trying to accomplish here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-340214704,https://api.github.com/repos/pydata/xarray/issues/1666,340214704,MDEyOklzc3VlQ29tbWVudDM0MDIxNDcwNA==,1217238,2017-10-28T19:36:07Z,2017-10-28T19:36:07Z,MEMBER,"You can iterate through a rolling object as a generator, e.g.,
```
In [6]: for k, v in r:
...: print(k.item(), v.values)
...:
0 [ True]
1 [0 1]
2 [1 2]
3 [2 3]
4 [3 4]
5 [4 5]
6 [5 6]
7 [6 7]
8 [7 8]
9 [8 9]
```
Does this help?
Note that something weird seems to be going on with the first time window. I don't know why it's a boolean. @jhamman any idea what's going on?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904