html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1666#issuecomment-438749995,https://api.github.com/repos/pydata/xarray/issues/1666,438749995,MDEyOklzc3VlQ29tbWVudDQzODc0OTk5NQ==,7441788,2018-11-14T17:35:17Z,2018-11-14T17:38:31Z,CONTRIBUTOR,"Also, the following code seems to accomplish the same as the above:
```
def apply_func_rolling(func, *args, **kwargs):
# determine rolling parameters, and remove them from kwargs
apply_func_kwargs = {'input_core_dims', 'output_core_dims', 'vectorize', 'join', 'dataset_join',
'keep_attrs', 'exclude_dims', 'dataset_fill_value', 'kwargs', 'dask',
'output_dtypes', 'output_sizes'}
min_periods = kwargs.pop('min_periods', None)
center = kwargs.pop('center', False)
dim = xr.core.utils.either_dict_or_kwargs(kwargs.pop('dim', None),
{k: v for k, v in kwargs.items()
if k not in apply_func_kwargs},
'apply_func_rolling')
if len(dim) != 1:
raise ValueError(""precisely one rolling dimension must be specified"")
rolling_dim = list(dim.keys())[0]
kwargs.pop(rolling_dim)
temp_rolling_dim = '__temp__{}__'.format(rolling_dim)
# change input_core_dims rolling_dim values to temp_rolling_dim
input_core_dims = kwargs.get('input_core_dims', None)
if input_core_dims:
kwargs['input_core_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_)
for dim_ in dims_] for dims_ in input_core_dims]
# change exclude_dims rolling_dim values to temp_rolling_dim
exclude_dims = kwargs.get('exclude_dims', None)
if exclude_dims:
kwargs['exclude_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_)
for dim_ in dims_] for dims_ in exclude_dims]
# call apply_func() with rolling-constructed objects
return xr.apply_ufunc(func,
*[(arg.rolling(dim=dim, min_periods=min_periods, center=center).
construct(temp_rolling_dim) if (rolling_dim in arg.dims) else arg)
for arg in args],
**kwargs)
apply_func_rolling(lambda a, b, w: ...,
variables, observations, weights,
date=N,
input_core_dims=[['date', 'dim1', 'dim2', 'var'],
['date', 'dim1', 'dim2'],
['date', 'dim1', 'dim2']],
output_core_dims=[['var']],
vectorize=True)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438372589,https://api.github.com/repos/pydata/xarray/issues/1666,438372589,MDEyOklzc3VlQ29tbWVudDQzODM3MjU4OQ==,7441788,2018-11-13T17:56:17Z,2018-11-13T20:16:57Z,CONTRIBUTOR,"> construct method does not allocate that large array in memory. It uses the strided trick and therefore consumers only the order of 1000x1000x1000.
Ah. I didn't realize that. Good to know.
What I'm actually looking to do is a rolling weighted regression. I have three DataArrays:
- observations, dims=('date', 'dim1', 'dim2')
- variables, dims=('date', 'dim1', 'dim2', 'var')
- weights, dims=('date', 'dim1', 'dim2')
I want to calculate a regression_coefficients DataArray with dims=('date', 'var'), where for each date it has the weighted regression coefficients calculated over the trailing N dates (over 'dim1' and 'dim2'). One way would be to put the three DataArrays in a Dataset, and then use a newly-defined `Dataset.rolling().apply()`. Another way would be to use an enhanced version of `apply_ufunc()` that can take `Rolling` objects. But now that I know that `DataArrayRolling.construct()` won't kill my machine, I'll try `apply_ufunc()` with the three `DataArrayRolling.construct()` objects. I'd welcome other suggestions.
OK, I seem to have got my problem working using:
```
apply_ufunc(lambda a, b, w: ...,
variables.rolling(date=N).construct('temp_date'),
observations.rolling(date=N).construct('temp_date'),
weights.rolling(date=N).construct('temp_date'),
input_core_dims=[['temp_date', 'dim1', 'dim2', 'var'],
['temp_date', 'dim1', 'dim2'],
['temp_date', 'dim1', 'dim2']],
output_core_dims=[['var']],
vectorize=True)
```
Still, I wonder if there isn't a more ""natural"" way of accomplishing this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438351586,https://api.github.com/repos/pydata/xarray/issues/1666,438351586,MDEyOklzc3VlQ29tbWVudDQzODM1MTU4Ng==,7441788,2018-11-13T17:06:38Z,2018-11-13T17:06:38Z,CONTRIBUTOR,"I think there are actually a couple different ways `Rolling.apply` could work, but this seems like one possible way:
```
from xarray.core.utils import maybe_wrap_array
from xarray.core.combine import concat
def rolling_apply(rolling, func, *args, **kwargs):
applied = [maybe_wrap_array(label, func(arr, *args, **kwargs)) for label, arr in rolling]
combined = concat(applied, dim=rolling.obj.coords[rolling.dim])
return combined
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438345386,https://api.github.com/repos/pydata/xarray/issues/1666,438345386,MDEyOklzc3VlQ29tbWVudDQzODM0NTM4Ng==,7441788,2018-11-13T16:50:08Z,2018-11-13T16:52:13Z,CONTRIBUTOR,The problem I have with `Rolling.construct` is the same that I have with `Rolling.reduce`: it's very (potentially) memory-inefficient. E.g. consider a 1000x1000x1000 array for which I want to apply rolling window of length 500 along the final dimension; I believe `Rolling.construct/reduce` will construct a 1000x1000x1000x500 array. This can quickly get out of hand.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438344608,https://api.github.com/repos/pydata/xarray/issues/1666,438344608,MDEyOklzc3VlQ29tbWVudDQzODM0NDYwOA==,7441788,2018-11-13T16:48:09Z,2018-11-13T16:48:09Z,CONTRIBUTOR,"Separately, maybe `apply_ufunc` should accepts `Rolling` objects?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904
https://github.com/pydata/xarray/issues/1666#issuecomment-438322013,https://api.github.com/repos/pydata/xarray/issues/1666,438322013,MDEyOklzc3VlQ29tbWVudDQzODMyMjAxMw==,7441788,2018-11-13T16:06:24Z,2018-11-13T16:06:24Z,CONTRIBUTOR,I think what is needed are `DataArrayRolling.apply` and `DatasetRolling.apply` (like `DataArrayGroupBy.apply` and `DatasetGroupBy.apply`). The problem with the `reduce` methods is that they are memory-inefficient.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,269297904