issue_comments

Also, the following code seems to accomplish the same as the above:

``` def apply_func_rolling(func, args, kwargs): # determine rolling parameters, and remove them from kwargs apply_func_kwargs = {'input_core_dims', 'output_core_dims', 'vectorize', 'join', 'dataset_join', 'keep_attrs', 'exclude_dims', 'dataset_fill_value', 'kwargs', 'dask', 'output_dtypes', 'output_sizes'} min_periods = kwargs.pop('min_periods', None) center = kwargs.pop('center', False) dim = xr.core.utils.either_dict_or_kwargs(kwargs.pop('dim', None), {k: v for k, v in kwargs.items() if k not in apply_func_kwargs}, 'apply_func_rolling') if len(dim) != 1: raise ValueError("precisely one rolling dimension must be specified") rolling_dim = list(dim.keys())[0] kwargs.pop(rolling_dim) temp_rolling_dim = 'temp{}__'.format(rolling_dim) # change input_core_dims rolling_dim values to temp_rolling_dim input_core_dims = kwargs.get('input_core_dims', None) if input_core_dims: kwargs['input_core_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_) for dim_ in dims_] for dims_ in input_core_dims] # change exclude_dims rolling_dim values to temp_rolling_dim exclude_dims = kwargs.get('exclude_dims', None) if exclude_dims: kwargs['exclude_dims'] = [[(temp_rolling_dim if (dim_ == rolling_dim) else dim_) for dim_ in dims_] for dims_ in exclude_dims] # call apply_func() with rolling-constructed objects return xr.apply_ufunc(func, [(arg.rolling(dim=dim, min_periods=min_periods, center=center). construct(temp_rolling_dim) if (rolling_dim in arg.dims) else arg) for arg in args], **kwargs)

apply_func_rolling(lambda a, b, w: ..., variables, observations, weights, date=N, input_core_dims=[['date', 'dim1', 'dim2', 'var'], ['date', 'dim1', 'dim2'], ['date', 'dim1', 'dim2']], output_core_dims=[['var']], vectorize=True) ```

construct method does not allocate that large array in memory. It uses the strided trick and therefore consumers only the order of 1000x1000x1000.

Ah. I didn't realize that. Good to know.

What I'm actually looking to do is a rolling weighted regression. I have three DataArrays: - observations, dims=('date', 'dim1', 'dim2') - variables, dims=('date', 'dim1', 'dim2', 'var') - weights, dims=('date', 'dim1', 'dim2')

I want to calculate a regression_coefficients DataArray with dims=('date', 'var'), where for each date it has the weighted regression coefficients calculated over the trailing N dates (over 'dim1' and 'dim2'). One way would be to put the three DataArrays in a Dataset, and then use a newly-defined Dataset.rolling().apply(). Another way would be to use an enhanced version of apply_ufunc() that can take Rolling objects. But now that I know that DataArrayRolling.construct() won't kill my machine, I'll try apply_ufunc() with the three DataArrayRolling.construct() objects. I'd welcome other suggestions.

OK, I seem to have got my problem working using: apply_ufunc(lambda a, b, w: ..., variables.rolling(date=N).construct('temp_date'), observations.rolling(date=N).construct('temp_date'), weights.rolling(date=N).construct('temp_date'), input_core_dims=[['temp_date', 'dim1', 'dim2', 'var'], ['temp_date', 'dim1', 'dim2'], ['temp_date', 'dim1', 'dim2']], output_core_dims=[['var']], vectorize=True) Still, I wonder if there isn't a more "natural" way of accomplishing this.

I think there are actually a couple different ways Rolling.apply could work, but this seems like one possible way:

``` from xarray.core.utils import maybe_wrap_array from xarray.core.combine import concat

def rolling_apply(rolling, func, args, kwargs): applied = [maybe_wrap_array(label, func(arr, args, **kwargs)) for label, arr in rolling] combined = concat(applied, dim=rolling.obj.coords[rolling.dim]) return combined ```

The problem I have with Rolling.construct is the same that I have with Rolling.reduce: it's very (potentially) memory-inefficient. E.g. consider a 1000x1000x1000 array for which I want to apply rolling window of length 500 along the final dimension; I believe Rolling.construct/reduce will construct a 1000x1000x1000x500 array. This can quickly get out of hand.

Separately, maybe apply_ufunc should accepts Rolling objects?

I think what is needed are DataArrayRolling.apply and DatasetRolling.apply (like DataArrayGroupBy.apply and DatasetGroupBy.apply). The problem with the reduce methods is that they are memory-inefficient.