issue_comments

There was some discussion about syntax at the dev meeting today. There were multiple votes in favour of allowing full control of padding in the rolling object itself. So we could have

1. .rolling(time=5, x=3, pad={"x": {"mode": "wrap"}, "time": False}), OR
2. .rolling(time=5, x=3).pad({"x": {"mode": "wrap"}, "time": False})

FWIW, I think 1 would be more efficient (but perhaps harder to implement). With this one, the DataArrayRolling object is only created once, with all of the required information.

If 2 were implemented, what should the return value of the .pad() call be? Would it call construct on the rolling object and return a DataArray? Or would it return an updated DataArrayRolling object? Or would it be a new type entirely (e.g., DataArrayPaddedRolling)?

Question: instead of adding stride to reduce and _reduce_method, why not add it as a member of DataArrayRolling directly? This would allow, e.g., __iter__ to use it as well, and seems like a cleaner interface.

I've been confused why some parameters are available only in construct (stride, fill_value), some are available both in construct and in the DataArrayRolling constructor (keep_attrs), and some are only available in the constructor (min_periods, center, and soon pad).

Mentioned in one of the comments above, but I think I've reached about the amount of time that I can spend on this right now. If there are other minor changes, please do let me know. I can also back out the breaking change if desired (although that will probably take some commit surgery).

The code also introduces a problem, the following no longer works: xr.DataArray([1, 2, 3], coords=dict(x=[1, 1, 1])).rolling(x=3).mean() thus you may have to use isel instead of sel if possible. Therefore I was also unable to test the following (from #2007 (comment)):

monthly.pad(month=n_months, mode="wrap").rolling(center=True, month=n_months, pad=False).mean(skipna=False)

Fixed and added a test for this. The example at the bottom now works.

@dcherian Thank you for reviewing. I've started working through your comments.

Also, FWIW, the test failure was caused by a problem in zarr/fsspec (https://github.com/intake/filesystem_spec/issues/707), which is fixed in master on fsspec (https://github.com/intake/filesystem_spec/pull/710).

So it should be fixed here whenever fsspec makes a release.

@dcherian Okay, I think this is in good shape. I added some more tests, and fixed a few more bugs. Most of the fixes have been squashed back down to the original commit.

I left the second commit separate for now because it's breaking. Previously, iterating over a rolling window returned only returned blocks present in the original array, and ignored the chunk of nans before the start or after the end of the array. This meant that the view of the DataArray or Dataset that was returned for each iteration was potentially a different size.

Here, instead, the iterator was changed so that each returned view matches the corresponding slice of the output of the construct() function. To me, this seems more intuitive, and makes it easier to develop algorithms without having to call construct.

Other than the fact that it's breaking, the main drawback (and difference with construct()) is that the windows themselves are labeled with coordinates along the rolling axis, except for any nan values before the start or after the end of the actual data, which do not have labels. For comparison, in construct, the window dimension does not have any coordinates associated with it by default.

After writing this, I'm wondering if it might be useful to simply drop the coordinates along the rolling axis, so that the behavior matches the behavior of construct() even more closely.

I'm open to thoughts/comments/criticisms/suggestions/questions/whatever.

@dcherian Thanks. I thought I was done, but I'm finding things that aren't working as expected. I'm working on adding more tests for expected behavior, so marking as WIP. Will ping back here when I'm finished (maybe today, maybe early next week).

FWIW, the test failures are all the same error, which should be fixed, but should have nothing to do with this PR.

I added support for .rolling(..., pad=False) in #5603. The basic implementation was simple, but getting it working for bottleneck/dask took a little more work.

That fixes, e.g., #4743, but I don't think it's a complete fix for this issue.

.pad(...).rolling(..., pad=False)

For this API, it seems that the only thing that would need to be implemented would be adding a pad keyword argument to rolling, defaulting to True. Is that correct?

@dcherian added. Let me know if you want any changes.

Changed to simply add examples to DataArrayRolling.reduce(). I changed the original description accordingly.

Actually, every window has at least 1 valid value, so this change isn't really needed at all.

I'll remove the code changes, and change the added example to show a working example of what I wanted.

I will say that min_periods isn't the most obvious name for this parameter--I'm struggling to figure out how the number of valid observations in a window is related to periods or periodicity.

Would it be reasonable to deprecate min_periods and change it to something like min_observations?

@mathause, thank you, I didn't realize that. I'll make those changes.

No worries--thanks for merging!

Rebased to fix the NEWS conflict.

@shoyer I force pushed the requested changes. Please let me know if there is anything else. Thanks!

I agree that it makes sense to recompute dimensions from variables.

Okay, that's what this PR already did (via the call to _replace_with_new_dims).

Rebased on master. From my perspective, this should be good to go.

@shoyer I'm sure you're busy... wondering if you might have time to review this (or direct it to someone else to review)? Other than my question above, it should be pretty straightforward.

I was looking for a way to drop dimensions, similar to the OP, and found this issue. I created an implementation of Dataset.drop_dims in #2767 which should work.

I was looking for this functionality and ran across #1949.

There was one bit of behavior that was unclear to me. Say I have

```python In [6]: data = xr.Dataset({'A': (['x', 'y'], np.random.randn(2, 3)), ...: 'B': ('x', np.random.randn(2)), ...: 'x': ['a', 'b'], 'z': np.pi})

In [7]: data Out[7]: <xarray.Dataset> Dimensions: (x: 2, y: 3) Coordinates: * x (x) <U1 'a' 'b' Dimensions without coordinates: y Data variables: A (x, y) float64 -0.662 -0.705 -0.1866 -0.8655 0.5816 -0.2308 B (x) float64 -0.1002 2.402 z float64 3.142

In [8]: data.drop_dims('x') Out[8]: <xarray.Dataset> Dimensions: () Data variables: z float64 3.142 ```

In this case, I dropped x, but because that eliminated variable A, I also dropped my y dimension, since A was the only variable using y.

At one point, I implemented this slightly differently, which instead gave

python In [8]: data.drop_dims('x') Out[8]: <xarray.Dataset> Dimensions: (y: 3) Dimensions without coordinates: y Data variables: z float64 3.142

In some ways, this seems preferable, especially if y has coordinates. The only downside I can see is that this version called Dataset._replace_vars_and_dims(), the docstring for which says it's deprecated (although it's still widely used).