In it goes. Thanks @darothen!

We just bumped numpy to 1.11, but 1.12 would be too new.

Let's just add a np.flip backport to core/npcompat.py. The whole function is only a couple of lines.

@jhamman Ohhh i totally misunderstood the last readout from travis-ci. Dealing with the scipy dependency is easy enough. ~However, another test fails because it uses np.flip() which wasn't added to numpy until v1.12.0. Do we want to bump the numpy version in the dependencies? Or is there another aproach to take here?~

Nevermind, easy solution is just to use other axis-reversal methods :)

@darothen - almost there. Two or three more dependency conflicts in the tests.

@jhamman done - caught me right while I was compiling GEOS-Chem, and the merge conflicts were very simple.

Thanks @darothen - can you resolve the merge conflicts?

@jhamman Think we're good. I deferred 4 small pep8 issues because they're in parts of the codebase which I don't think I ever touched, and i'm worried they're going to screw up the merge.

@darothen - we have a few other PRs to wrap up for 0.10 so end of the week is okay.

@jhamman Gotcha, I'll clean everything up by the end of the week. If that's going to block 0.10.0, let me know and I'll shuffle some things around to prioritize this.

@shoyer fixed.

Hmmm. Something is really screwy with my feature branch and making the task of cleaning up the merge difficult. I'll work on fixing this.

Try refreshing? Latest commit is 7a767d8 and has all these changes plus some more tweaks.

Daniel Rothenberg * Postdoctoral Research Associate Center for Global Change Science Massachusetts Institute of Technology A: 77 Massachusetts Ave | E18-402A Cambridge, MA 02139 T: (502) 648-7513; T: (617) 258-0407 E: darothen@mit.edu H: *www.danielrothenberg.com http://github.com/darothen [image: http://www.linkedin.com/in/rothenbergdaniel/] http://www.linkedin.com/in/rothenbergdaniel/ http://www.twitter.com/danrothenberg

On Tue, Sep 12, 2017 at 12:02 PM, Stephan Hoyer notifications@github.com wrote:

@shoyer commented on this pull request.

In xarray/core/resample.py https://github.com/pydata/xarray/pull/1272#discussion_r138390791:

• f = interp1d(x, y, kind=kind, axis=axis, bounds_error=True,
• assume_sorted=True)

• Prepare coordinates by dropping non-dimension coordinates along the

• resampling dimension.

• note: the lower-level Variable API could be used to speed this up

• coords = OrderedDict()
• if self._obj.data_vars:
• var = list(self._obj.data_vars.keys())[0]
• da = self._obj[var].copy()

did you push those commits yet? I'm not seeing it yet

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/1272#discussion_r138390791, or mute the thread https://github.com/notifications/unsubscribe-auth/AEwtqKtNFFYYlmBgLHyDLPVZNJe7T18Gks5shqqdgaJpZM4MDalB .

Resolved to drop auxiliary coordinates which are defined along the dimension to be re-sampled. This makes sense; if someone wants them to be interpolated or manipulated in some way, then they should promote them from coordinates to variables before doing the resampling.

In response to #1328, count() works just fine if you call it from a Resample object. Works for both resampling and up-sampling, but it will preserve the shape of the non-resampled dimensions. I think that's fine, because count() treats NaN as missing by default, so you can immediately know in which grid cells you have missing data :)

Final review, @shoyer, before merging in anticipation of 0.10.0?

I've opted not to include a chart outlining the various upsampling options...

This is OK by me

All set except for my one question to @shoyer above. I've opted not to include a chart outlining the various upsampling options... couldn't really think of a nice and clean way to do so, because adding it to the time series doc page ends up being really ugly and there isn't quite enough substance for its own worked example page.

Okay, it was a bit of effort but I implemented upsampling. For the padding methods I just re-index the Dataset or DataArray using the re-sampled time frequencies. I also added interpolation, but that was a bit tricky; we have to sort of break the split-apply-combine idiom to do that, so I created a Resampler mix-in which could contain the logic for the up-sampling. The DatasetResampler and DataArrayResampler each then implement similar logic for doing the interpolation. The up-sampling is designed to work with n-dimensional data.

The padding methods work 100% with dask arrays - since we're just calling xarray methods which themselves work with dask arrays! There are some eager computations (just the calculation of the up-sampled time frequencies) but I don't think that's a major issue; the actual re-indexing/padding is deferred. Interpolation works with dask arrays too, but eagerly does the computations.

Could use a review from @shoyer or @jhamman.

New TODO list:

• [ ] Add example chart to the timeseries doc page comparing the different upsampling options
• [x] Additional up-sampling test cases for both DataArrays and Datasets
• [x] Code clean-up
• [x] What's new

I like your approach here! Just some various comments on the implementation.

TODO

• [x] ensure that count() works on Data{Array,set}Resample objects
• [x] refactor Data{Array,set}Resample objects into a stand-alone file core/resample.py alongside core/groupby.py
• [x] wrap pytest.warns around tests targeting old API
• [x] move old API tests into stand-alone
• [x] Crude up-sampling. Copy/pasting Stephan's earlier comment from Feb 20:

I think we need to fix this before merging this PR, since it suggests the existing functionality would only exist in deprecated form. Pandas does this with a method called .asfreq, though this is basically pure sugar since in practice I think it works exactly the same as .first (or .mean if only doing pure upsampling).

Alright @jhamman, here's the complete list of work left here. I'll tackle some of it during my commutes this week.

Finished off everything except upsampling. In pandas, all upsampling works by constructing a new time index (which we already do) and then filling in the NaNs that result in the dataset with one of a few different rules. Not sure how involved this will be, but I anticipate this can all be implemented in core/resample.py

I did my best to re-base everything to master... plan on spending an hour or so figuring out what's broken and at least restoring the status quo.

I think a pull against the new releases is critical to see what breaks. Beyond that, just code clean up and testing. I can try to bump this higher on my priority list.

@darothen - can you give a summary of what's left to do here?

I can't really find many examples of people using this as a substitute for time group-bys... it seems that's what the pd.TimeGrouper is for, in conjunction with a normal .groupby().

I think this is mostly because TimeGrouper has been around far longer than non-aggregating resample.

Should .apply() really work on non-aggregation functions? Based on the pandas documentation it seems like "resample" is truly just a synonym for a transformation of the time dimension. I can't really find many examples of people using this as a substitute for time group-bys... it seems that's what the pd.TimeGrouper is for, in conjunction with a normal .groupby().

As written, non-aggregation ("transformation"?) doesn't work because the call in _combine() to _maybe_reorder() messes things up (it drops all of the data along the resampled dimension). It shouldn't be too hard to fix this, although I'm leaning more and more to making stand-alone Data{Array,set}Resample classes in a separate file which only loosely inherit from their Data{Array,set}GroupBy cousins, since they need to re-write some really critical parts of the underlying machinery.

Thanks for the feedback, @shoyer! Will circle back around to continue working on this in a few days when I have some free time.

• Daniel

The only issue now is the signature for init() in Data{set,Array}Resample, where we have to add in two keyword arguments. Python 2.x doesn't like named arguments after args. There are a few options here, mostly just playing with *kwargs as in this StackOverflow thread.

Yes, use pop, e.g., dim = kwargs.pop('dim', None). pop removes the arguments from kwargs, so you can pass on the remaining ones unchanged to the super class method.

Smoothed out most of the problems from earlier and missing details. Still not sure if it's wise to refactor most of the resampling logic into a new resample.py, like what was done with rolling, but it still makes some sense to keep things in groupby.py because we're just subclassing existing machinery from there.

The only issue now is the signature for init() in Data{set,Array}Resample, where we have to add in two keyword arguments. Python 2.x doesn't like named arguments after *args. There are a few options here, mostly just playing with **kwargs as in this StackOverflow thread.