html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/2104#issuecomment-395621709,https://api.github.com/repos/pydata/xarray/issues/2104,395621709,MDEyOklzc3VlQ29tbWVudDM5NTYyMTcwOQ==,1197350,2018-06-08T02:00:11Z,2018-06-08T02:00:11Z,MEMBER,Congratulations on this great new feature!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-395608335,https://api.github.com/repos/pydata/xarray/issues/2104,395608335,MDEyOklzc3VlQ29tbWVudDM5NTYwODMzNQ==,6815844,2018-06-08T00:33:56Z,2018-06-08T00:33:56Z,MEMBER,"Merged! Thanks, @shoyer, @fmaussion, @jhamman, for the detailed review.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-395596744,https://api.github.com/repos/pydata/xarray/issues/2104,395596744,MDEyOklzc3VlQ29tbWVudDM5NTU5Njc0NA==,1217238,2018-06-07T23:24:58Z,2018-06-07T23:24:58Z,MEMBER,"OK, let's merge this when CI passes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-395595931,https://api.github.com/repos/pydata/xarray/issues/2104,395595931,MDEyOklzc3VlQ29tbWVudDM5NTU5NTkzMQ==,6815844,2018-06-07T23:20:35Z,2018-06-07T23:20:35Z,MEMBER,"Thanks, @shoyer. > I think the last thing to do here is remove keep_attrs? Done! (I thought I reverted this change already...)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-395243708,https://api.github.com/repos/pydata/xarray/issues/2104,395243708,MDEyOklzc3VlQ29tbWVudDM5NTI0MzcwOA==,6815844,2018-06-06T23:18:25Z,2018-06-06T23:18:25Z,MEMBER,`interp_like` sounds a good idea. I like it in another PR as this is already huge.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-395169022,https://api.github.com/repos/pydata/xarray/issues/2104,395169022,MDEyOklzc3VlQ29tbWVudDM5NTE2OTAyMg==,1217238,2018-06-06T18:30:42Z,2018-06-06T18:30:42Z,MEMBER,Do you want to add `interp_like` here or should we leave it for a later PR?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-394939096,https://api.github.com/repos/pydata/xarray/issues/2104,394939096,MDEyOklzc3VlQ29tbWVudDM5NDkzOTA5Ng==,1217238,2018-06-06T04:49:22Z,2018-06-06T04:49:22Z,MEMBER,"Hmm. I guess numeric_only may currently only be an argument for the reduce method. We probably should add it for methods like mean, too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-394932923,https://api.github.com/repos/pydata/xarray/issues/2104,394932923,MDEyOklzc3VlQ29tbWVudDM5NDkzMjkyMw==,6815844,2018-06-06T04:00:18Z,2018-06-06T04:00:18Z,MEMBER,"Thanks, @shoyer > We do the same thing for aggregations like mean(), behavior originally copied from pandas: http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.DataFrame.mean.html Thanks. I have not been surprised when `mean` drops the object type array. So it would be OK also for `interp`. Maybe this PR is ready, then. BTW, I noticed that our `mean()` does not accept the `numeric_only` keyword argument. Our docs do not say about `numeric_only` keyword either. Is it intended? (This behavior looks already accepted, and I think it is OK if we do not support this.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-394754176,https://api.github.com/repos/pydata/xarray/issues/2104,394754176,MDEyOklzc3VlQ29tbWVudDM5NDc1NDE3Ng==,1217238,2018-06-05T15:30:23Z,2018-06-05T15:30:23Z,MEMBER,"> Do we have other methods that behave similarly? We do the same thing for aggregations like `mean()`, behavior originally copied from pandas: http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.DataFrame.mean.html We could add a `numeric_only=True` argument (like we do for reduction methods) to make this behavior more obvious and allow it to be controlled.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-394532951,https://api.github.com/repos/pydata/xarray/issues/2104,394532951,MDEyOklzc3VlQ29tbWVudDM5NDUzMjk1MQ==,6815844,2018-06-04T23:33:42Z,2018-06-05T00:56:41Z,MEMBER,"Done! But, I am slightly worrying if silently dropping non-numeric array is surprising. Do we have other methods that behave similarly? Or maybe we can raise a warning?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-393770344,https://api.github.com/repos/pydata/xarray/issues/2104,393770344,MDEyOklzc3VlQ29tbWVudDM5Mzc3MDM0NA==,1217238,2018-06-01T06:05:07Z,2018-06-01T06:05:07Z,MEMBER,"I would wait before trying to do interpolation with datetime types. There might be use cases for this but I can't think of them offhand. `resample()` would seem to cover most of these. On the other hand, it could indeed be interesting to extend `resample()` to handling numbers. I proposed this a while ago for pandas (https://github.com/pandas-dev/pandas/issues/12828).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-393761310,https://api.github.com/repos/pydata/xarray/issues/2104,393761310,MDEyOklzc3VlQ29tbWVudDM5Mzc2MTMxMA==,6815844,2018-06-01T05:20:59Z,2018-06-01T05:20:59Z,MEMBER,"> 1. I suppose it probably does make sense to interpolate numeric coordinates, too. So this probably doesn't need to change. Agreed. > 2. I do think we should drop non-numeric variables. There just isn't any way to meaningfully interpolate them (especially strings). This means we will probably need to add a check in DataArray.interp to ensure that the variable dtype is numeric and otherwise raise an error. Also agreed. In theory, datetime dtype could be safely interpolated. Maybe we can support this if there is a demand. It might be necessary to consider the same design issue also for `interpolate_na`, `bfill`, `ffill`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-393757211,https://api.github.com/repos/pydata/xarray/issues/2104,393757211,MDEyOklzc3VlQ29tbWVudDM5Mzc1NzIxMQ==,1217238,2018-06-01T04:51:39Z,2018-06-01T04:51:39Z,MEMBER,"My opinions: 1. I suppose it probably does make sense to interpolate numeric coordinates, too. So this probably doesn't need to change. 2. I do think we should drop non-numeric variables. There just isn't any way to meaningfully interpolate them (especially strings). This means we will probably need to add a check in `DataArray.interp` to ensure that the variable dtype is numeric and otherwise raise an error.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-393756834,https://api.github.com/repos/pydata/xarray/issues/2104,393756834,MDEyOklzc3VlQ29tbWVudDM5Mzc1NjgzNA==,1217238,2018-06-01T04:48:33Z,2018-06-01T04:48:33Z,MEMBER,"This is very elegant and I think very close to being ready to merge. There are two design issues that I would like to discuss first: 1. Should we interpolate both `data_vars` and `coords` or only `data_vars`? 2. How should we handle non-numeric variables? The current behavior of this PR is: 1. Interpolate all variables, both `data_vars` and `coords`. 2. Non-numeric variables raise an error, because they can't be converted into float. It might make sense to copy the behavior from `Dataset.reduce`: 1. Only interpolate `data_vars`. 2. Drop non-numeric variables instead of trying to interpolate them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-393032031,https://api.github.com/repos/pydata/xarray/issues/2104,393032031,MDEyOklzc3VlQ29tbWVudDM5MzAzMjAzMQ==,6815844,2018-05-30T05:08:12Z,2018-05-30T05:08:12Z,MEMBER,"Thanks for the careful review. All done.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-391402735,https://api.github.com/repos/pydata/xarray/issues/2104,391402735,MDEyOklzc3VlQ29tbWVudDM5MTQwMjczNQ==,1217238,2018-05-23T16:03:06Z,2018-05-23T16:03:06Z,MEMBER,"Sorry, I still need to find time to finish reviewing this — hopefully very soon! On Wed, May 23, 2018 at 8:57 AM Joe Hamman wrote: > Is there more to do here or is this ready to go in? Seems to me its ready > to go. > > — > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-391400628,https://api.github.com/repos/pydata/xarray/issues/2104,391400628,MDEyOklzc3VlQ29tbWVudDM5MTQwMDYyOA==,2443309,2018-05-23T15:56:57Z,2018-05-23T15:56:57Z,MEMBER,Is there more to do here or is this ready to go in? Seems to me its ready to go. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-389680743,https://api.github.com/repos/pydata/xarray/issues/2104,389680743,MDEyOklzc3VlQ29tbWVudDM4OTY4MDc0Mw==,1217238,2018-05-16T22:05:14Z,2018-05-16T22:05:14Z,MEMBER,"> Very nice! I noticed that the interpolation is performed among dimensions rather than coordinates in this PR. However the limitation to that is interpolation to/from curvilinear grids is not supported, which I think is a common enough use case, and would be extremely nice to have. Pretty sure scipy's interpolation tools work out of the box with curvilinear grids. Is an updated interface which works on coordinate variables rather than dimensions planned? Yes, our intent is to eventually extend this same interface to handle these cases, but not in this initial PR.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388602254,https://api.github.com/repos/pydata/xarray/issues/2104,388602254,MDEyOklzc3VlQ29tbWVudDM4ODYwMjI1NA==,6815844,2018-05-13T05:15:53Z,2018-05-13T05:15:53Z,MEMBER,I think this is ready for final review,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388219645,https://api.github.com/repos/pydata/xarray/issues/2104,388219645,MDEyOklzc3VlQ29tbWVudDM4ODIxOTY0NQ==,2448579,2018-05-10T23:53:32Z,2018-05-10T23:53:32Z,MEMBER,"@fujiisoup That illustration is great. Thanks for all this great work. Yes, adding more information to that would work well.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388217237,https://api.github.com/repos/pydata/xarray/issues/2104,388217237,MDEyOklzc3VlQ29tbWVudDM4ODIxNzIzNw==,6815844,2018-05-10T23:37:26Z,2018-05-10T23:37:26Z,MEMBER,"> I think it would be useful to have a table in the docs showing the differences between / when to use interp, interpolate_na, reindex, sel(..., method='nearest'), resample. Agreed. Thanks for the nice suggestion :+1: We have discussed a cheat sheet #1552 previously, but no work has been done yet. I added a simple [illustration](https://github.com/fujiisoup/xarray/blob/b3c76d7fec45e912b5ff688b5ceb74467cab1e51/doc/_static/advanced_selection_interpolation.svg) presenting the difference between advanced-`sel` and -`interp` for this addition, and I think we can enrich this kind of figure to clarify the differences among these set of methods.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388215681,https://api.github.com/repos/pydata/xarray/issues/2104,388215681,MDEyOklzc3VlQ29tbWVudDM4ODIxNTY4MQ==,2448579,2018-05-10T23:27:27Z,2018-05-10T23:27:27Z,MEMBER,"I think it would be useful to have a table in the docs showing the differences between / when to use `interp`, `interpolate_na`, `reindex`, `sel(..., method='nearest')`, `resample`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388052465,https://api.github.com/repos/pydata/xarray/issues/2104,388052465,MDEyOklzc3VlQ29tbWVudDM4ODA1MjQ2NQ==,6815844,2018-05-10T13:26:06Z,2018-05-10T13:26:06Z,MEMBER,"@fmaussion , thanks for the example! It's a nicely simple but descriptive example. Updated docs. After your suggestion about attrs, I found a bug in `apply_ufunc` (#2114). The tests are added but now failing because of this issue.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388009361,https://api.github.com/repos/pydata/xarray/issues/2104,388009361,MDEyOklzc3VlQ29tbWVudDM4ODAwOTM2MQ==,10050469,2018-05-10T09:57:26Z,2018-05-10T09:57:26Z,MEMBER,"> Is there any problem if we always keep attrs in the interpolation? In my opinion it should be the default, yes.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388009096,https://api.github.com/repos/pydata/xarray/issues/2104,388009096,MDEyOklzc3VlQ29tbWVudDM4ODAwOTA5Ng==,10050469,2018-05-10T09:56:15Z,2018-05-10T09:56:15Z,MEMBER,"> BTW, do you have any idea about an example for the new interp feature? A quick shot would be to use the existing sample dataset: ```python import matplotlib.pyplot as plt import xarray as xr import numpy as np # Raw data ds = xr.tutorial.load_dataset('air_temperature') ds.air.isel(time=0).plot() plt.title('Raw data') # Interpolated data new_lon = np.linspace(ds.lon[0], ds.lon[-1], ds.dims['lon'] * 4) new_lat = np.linspace(ds.lat[0], ds.lat[-1], ds.dims['lat'] * 4) dsi = ds.interp(lon=new_lon, lat=new_lat) dsi.air.isel(time=0).plot() plt.title('Interpolated data') ``` Which produces the plots: ![index](https://user-images.githubusercontent.com/10050469/39864161-186fe04c-5449-11e8-84bb-b444eae21def.png) ![index2](https://user-images.githubusercontent.com/10050469/39864162-1872a070-5449-11e8-9992-32acf94031cc.png) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-388005058,https://api.github.com/repos/pydata/xarray/issues/2104,388005058,MDEyOklzc3VlQ29tbWVudDM4ODAwNTA1OA==,6815844,2018-05-10T09:37:34Z,2018-05-10T09:37:34Z,MEMBER,"Thank you for the double checking :+1: > it would be great to implement the keep_attrs kwarg as well, because interpolation often conserves the units of data. Agreed. Is there any problem if we always keep attrs in the interpolation? `.sel` always keeps attrs and I think it would be better if users have less choice in each method. BTW, do you have any idea about an example for the new interp feature? It would be nice if we have a good-looking example on the doc.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387993706,https://api.github.com/repos/pydata/xarray/issues/2104,387993706,MDEyOklzc3VlQ29tbWVudDM4Nzk5MzcwNg==,10050469,2018-05-10T08:48:12Z,2018-05-10T08:48:12Z,MEMBER,"> As the interpolation routine can also be used for non-uniform gridded data, I don't think passing np.arange(NN) to scipy is a good idea (it will change the result if higher order method such as 'cubic' is used). Right. Another feature of geospatial grids is that they are regularly spaced, which is not always the case here. Forget about my idea for now. I tested your updated branch and we now come to the same results with my test data! I took the liberty to edit your [comment](https://github.com/pydata/xarray/pull/2104#issuecomment-387914727) because in your current implementation ``assume_sorted`` is ``False`` per default. Another wish (sorry! ;): it would be great to implement the ``keep_attrs`` kwarg as well, because interpolation often conserves the units of data. Thanks!!!! ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387914727,https://api.github.com/repos/pydata/xarray/issues/2104,387914727,MDEyOklzc3VlQ29tbWVudDM4NzkxNDcyNw==,6815844,2018-05-10T00:29:12Z,2018-05-10T08:45:30Z,MEMBER,"Thanks, @fmaussion I didn't realize that scipy.interpolate.interpn does not sort the original coordinates (interp1d does). Thanks for pointing this out! > The implementation for you would be straightforward, you'd just have to pass np.arange(NN) to scipy instead of the actual coordinates. As the interpolation routine can also be used for non-uniform gridded data, I don't think passing np.arange(NN) to scipy is a good idea (it will change the result if higher order method such as 'cubic' is used). Instead, I would like to call `sortby` in our `interp` routine, so that the array passed to scipy is always sorted in the ascending order. The new API I would propose is ```python da.interp(x=[0, 1, 2], method='linear', assume_sorted=False, kwargs={'bounds_error': False, 'fill_value': 'extrapolate'}) ``` with a new keyword `assume_sorted`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387848141,https://api.github.com/repos/pydata/xarray/issues/2104,387848141,MDEyOklzc3VlQ29tbWVudDM4Nzg0ODE0MQ==,10050469,2018-05-09T19:24:27Z,2018-05-09T19:24:27Z,MEMBER,"@fujiisoup I gave a go at your branch this evening, trying a few things on real data cases. First of all: this is great! Really looking forward to see this in xarray. I noticed a small issue: currently this doesn't work with decreasing coordinates: ```python nx = 5 ny = 4 lon = np.arange(nx) lat = np.arange(ny)[::-1] data = np.arange(nx*ny).reshape((ny, nx)) da = xr.DataArray(data, dims=('lat', 'lon'), coords={'lon':lon, 'lat':lat}) da.interp(lon=[1.1, 2.2, 3.3], lat=[1.2, 2.3]) -> ValueError from SciPy: The points in dimension 0 must be strictly ascending ``` Traceback:
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) in () ----> 1 da.interp(lon=[1.1, 2.2, 3.3], lat=[2.3, 1.2]) ~/tmp/xarray-fuji/xarray/core/dataarray.py in interp(self, method, kwargs, **coords) 912 """""" 913 ds = self._to_temp_dataset().interp( --> 914 method=method, kwargs=kwargs, **coords) 915 return self._from_temp_dataset(ds) 916 ~/tmp/xarray-fuji/xarray/core/dataset.py in interp(self, method, kwargs, **coords) 1824 if name not in [k for k, v in indexers_list]: 1825 variables[name] = missing.interp( -> 1826 var, var_indexers, method, **kwargs) 1827 1828 coord_names = set(variables).intersection(self._coord_names) ~/tmp/xarray-fuji/xarray/core/missing.py in interp(obj, indexes_coords, method, **kwargs) 441 kwargs={'x': x, 'new_x': destination, 'method': method, 442 'kwargs': kwargs}, --> 443 keep_attrs=True) 444 445 if all(x1.dims == new_x1.dims for x1, new_x1 in zip(x, new_x)): ~/tmp/xarray-fuji/xarray/core/computation.py in apply_ufunc(func, *args, **kwargs) 934 keep_attrs=keep_attrs) 935 elif any(isinstance(a, Variable) for a in args): --> 936 return variables_ufunc(*args) 937 else: 938 return apply_array_ufunc(func, *args, dask=dask) ~/tmp/xarray-fuji/xarray/core/computation.py in apply_variable_ufunc(func, *args, **kwargs) 563 raise ValueError('unknown setting for dask array handling in ' 564 'apply_ufunc: {}'.format(dask)) --> 565 result_data = func(*input_data) 566 567 if signature.num_outputs > 1: ~/tmp/xarray-fuji/xarray/core/missing.py in interp_func(obj, x, new_x, method, kwargs) 499 500 func, kwargs = _get_interpolator_nd(method, **kwargs) --> 501 return _interpnd(obj, x, new_x, func, kwargs) 502 503 ~/tmp/xarray-fuji/xarray/core/missing.py in _interpnd(obj, x, new_x, func, kwargs) 518 # stack new_x to 1 vector, with reshape 519 xi = np.stack([x1.values.ravel() for x1 in new_x], axis=-1) --> 520 rslt = func(x, obj, xi, **kwargs) 521 # move back the interpolation axes to the last position 522 rslt = rslt.transpose(range(-rslt.ndim + 1, 1)) ~/.pyvirtualenvs/py3/lib/python3.5/site-packages/scipy/interpolate/interpolate.py in interpn(points, values, xi, method, bounds_error, fill_value) 2589 if not np.all(np.diff(p) > 0.): 2590 raise ValueError(""The points in dimension %d must be strictly "" -> 2591 ""ascending"" % i) 2592 if not np.asarray(p).ndim == 1: 2593 raise ValueError(""The points in dimension %d must be "" ValueError: The points in dimension 0 must be strictly ascending
Decreasing coordinates are really frequent in climate data (latitudes are often decreasing, don't ask me why), so I wondered how I implemented this in [salem](http://salem.readthedocs.io/en/latest/gis.html), which brings me to the second point: I think we need an ``interp()`` equivalent to ``isel()`` too, which is the one I could make use of with salem (and, maybe, several other georeferencing tools). The way salem works is that it uses the concept of geospatial grid where the points to interpolate are defined in a local coordinate system which is always positive and increasing. That is, if I want to use xarray's interpolation routines, I'd like to be able to tell xarray that I'd like to interpolate at ""indices [1.1, 2.1, 3.1]"" for example, where these can really be understood as indexes in the way ``da.isel()`` sees them. I don't know if we really need a ``i_interp`` for this though, or if it could be solved with a kwarg to the existing API. The implementation for you would be straightforward, you'd just have to pass ``np.arange(NN)`` to scipy instead of the actual coordinates. Thoughts? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387634881,https://api.github.com/repos/pydata/xarray/issues/2104,387634881,MDEyOklzc3VlQ29tbWVudDM4NzYzNDg4MQ==,10050469,2018-05-09T06:33:26Z,2018-05-09T06:33:26Z,MEMBER,"> I prefer not to support this now for keeping the interpolation rule simple. Agreed. Users can turn their 1D non dimension coordinates into dimension coordinates if they want to.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387583395,https://api.github.com/repos/pydata/xarray/issues/2104,387583395,MDEyOklzc3VlQ29tbWVudDM4NzU4MzM5NQ==,6815844,2018-05-09T00:29:36Z,2018-05-09T00:29:36Z,MEMBER,"> As I think I've mentioned before, I think we should save interpolation of unstructured grids for later -- if only to keep the complexity of this PR to the minimum. I agreed that it would be better to concentrate on structured interpolation only in this PR. I concerned about the interpolation along **1-dimensional** but non-dimensional coordinate, which @fmaussion suggested in a review [comment](https://github.com/pydata/xarray/pull/2104#discussion_r186745479). It is actually the structured interpolation, but currently, I do not support this just because of the analogy to `.sel()`. If we want to support this, we may need to decide its behavior in particular the dimension or coordinate of the interpolated array. I prefer not to support this now for keeping the interpolation rule simple.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387580695,https://api.github.com/repos/pydata/xarray/issues/2104,387580695,MDEyOklzc3VlQ29tbWVudDM4NzU4MDY5NQ==,1217238,2018-05-09T00:12:14Z,2018-05-09T00:12:14Z,MEMBER,"> should we support the interpolation along the non-dimensional coordinate? As I think I've mentioned before, I think we should save interpolation of unstructured grids for later -- if only to keep the complexity of this PR to the minimum. Interpolation on unstructured grids needs some of auxiliary data structure to be done efficiently, so it might make sense to wait until after we support flexible index types such as a KDTree index saved on xarray objects. That said, we could also add it now (immediately after this PR), and add support for caching KDTrees later. I think the API for unstructured grid interpolation is pretty straightforward and could even reuse the same `interp`/`interpolate_at` method. Unstructured grid interpolation would be triggered whenever multiple existing coordinates that share some of the same dimensions are interpolated. > what should be the default value of 'bounds_error' and fill_value. scipy's interpolator assumes bounds_error=True by default, but I think bounds_error=False is more xarray-like, but it may be unintuitive for users very familiar with scipy's behavior. I agree, by default we should switch to `bounds_error=False`, and replace extrapolated values with NaN, which is already the default fill value for `interp1d`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387576123,https://api.github.com/repos/pydata/xarray/issues/2104,387576123,MDEyOklzc3VlQ29tbWVudDM4NzU3NjEyMw==,6815844,2018-05-08T23:43:29Z,2018-05-08T23:46:17Z,MEMBER,"Thanks, @fmaussion The current design question is + should we support the interpolation along the non-dimensional coordinate? + what should be the default value of 'bounds_error' and `fill_value`. scipy's interpolator assumes `bounds_error=True` by default, but I think `bounds_error=False` is more xarray-like, but it may be unintuitive for users very familiar with scipy's behavior.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387388988,https://api.github.com/repos/pydata/xarray/issues/2104,387388988,MDEyOklzc3VlQ29tbWVudDM4NzM4ODk4OA==,6815844,2018-05-08T12:41:29Z,2018-05-08T12:41:57Z,MEMBER,"Thanks, @fmaussion. I think it is almost ready. The remaining things are + I am looking for a good-looking example for our docs on rtd. + Just before the merge, I'm going to rename missing.py to interp.py so that the filename is more descriptive. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387387039,https://api.github.com/repos/pydata/xarray/issues/2104,387387039,MDEyOklzc3VlQ29tbWVudDM4NzM4NzAzOQ==,10050469,2018-05-08T12:33:36Z,2018-05-08T12:33:36Z,MEMBER,Let us know when this is ready for review! I'm happy to have a look at it.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387386033,https://api.github.com/repos/pydata/xarray/issues/2104,387386033,MDEyOklzc3VlQ29tbWVudDM4NzM4NjAzMw==,10050469,2018-05-08T12:29:53Z,2018-05-08T12:29:53Z,MEMBER,"> Honestly, I am not very happy with kwargs={} arguments (it would be more beautiful if they can be keyword arguments), but I think it is a necessary pain I don't think it's *that* bad. It makes it clear that the arguments are passed over to the underlying function, and that xarray is not doing all the magic by itself.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-387295942,https://api.github.com/repos/pydata/xarray/issues/2104,387295942,MDEyOklzc3VlQ29tbWVudDM4NzI5NTk0Mg==,6815844,2018-05-08T06:13:49Z,2018-05-08T06:13:49Z,MEMBER,"> So I think it might make sense to save skipna=True for interpolation on unstructured grids. I agreed. I updated some of this PR. The proposed API is ```python da.interp(x=[0, 1, 2], method='linear', kwargs={'bounds_error': False, 'fill_value': 'extrapolate'}) ``` Honestly, I am not very happy with `kwargs={}` arguments (it would be more beautiful if they can be keyword arguments), but I think it is a necessary pain.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386938847,https://api.github.com/repos/pydata/xarray/issues/2104,386938847,MDEyOklzc3VlQ29tbWVudDM4NjkzODg0Nw==,1217238,2018-05-07T02:14:11Z,2018-05-07T02:14:11Z,MEMBER,"My inclination would be to limit this method to interpolation on regular grids, i.e., the methods supported by [scipy.interpolate.interpn](https://docs.scipy.org/doc/scipy-1.0.0/reference/generated/scipy.interpolate.interpn.html) or interp1d. I think a `skipna=True` option could be useful, but I don't know how we could do that for regular interpolation with multi-dimensional data. So I think it might make sense to save `skipna=True` for interpolation on unstructured grids.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386876701,https://api.github.com/repos/pydata/xarray/issues/2104,386876701,MDEyOklzc3VlQ29tbWVudDM4Njg3NjcwMQ==,6815844,2018-05-06T12:41:07Z,2018-05-06T12:41:07Z,MEMBER,"Yes, I am thinking this for multidimensional interpolation of nan-including arrays. (For 1-dimensional interpolation, we can interpolate them row by row) Although this should be another work (PR), I am wondering what the best API is. Is `skipna=True` option handy? If so, which should be the default?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386876126,https://api.github.com/repos/pydata/xarray/issues/2104,386876126,MDEyOklzc3VlQ29tbWVudDM4Njg3NjEyNg==,10050469,2018-05-06T12:31:27Z,2018-05-06T12:31:27Z,MEMBER,"> I think the addition of the linear interpolation is still nice, but it would be nicer if we could also implement nan-skipping interpolations, which will enable us to use higher order interpolation with nan-including arrays. Do you mean you want to use unstructured data interpolation routines? (https://docs.scipy.org/doc/scipy/reference/interpolate.html#multivariate-interpolation) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386874264,https://api.github.com/repos/pydata/xarray/issues/2104,386874264,MDEyOklzc3VlQ29tbWVudDM4Njg3NDI2NA==,6815844,2018-05-06T12:00:33Z,2018-05-06T12:00:33Z,MEMBER,"> RegularGridInterpolator handles NaNs pretty much the same way @shoyer describes it. I did not certainly realize how RegularGridInterpolator works, but it seems to work with nan including array too as you pointed out. But higher order interpolations (RegularGridInterpolator is `linear`), such as `RectBivariateSpline`, would not work (return an all nan array) when just 1 nan is included in the target array. It is the same with higher order interpolations in `scipy.interpolate.interp1d` such as `cubic`. > Why don't leave the NaN handling part to the actual function doing the interpolation? It is the original behavior of `scipy.interpolate` functions (except for `linear` or `nearest` methods), which do some matrix inversions in its algorithm. I think the addition of the linear interpolation is still nice, but it would be nicer if we could also implement nan-skipping interpolations, which will enable us to use higher order interpolation with nan-including arrays.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386869104,https://api.github.com/repos/pydata/xarray/issues/2104,386869104,MDEyOklzc3VlQ29tbWVudDM4Njg2OTEwNA==,10050469,2018-05-06T10:24:00Z,2018-05-06T10:24:00Z,MEMBER,"> linear and nearest can handle NaN intuitively, but other methods need to invert some matrices Why don't leave the NaN handling part to the actual function doing the interpolation? RegularGridInterpolator handles NaNs pretty much the same way @shoyer describes it.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386854560,https://api.github.com/repos/pydata/xarray/issues/2104,386854560,MDEyOklzc3VlQ29tbWVudDM4Njg1NDU2MA==,6815844,2018-05-06T05:13:09Z,2018-05-06T05:13:09Z,MEMBER,"@shoyer , Yes, `linear` and `nearest` can handle NaN intuitively, but other methods need to invert some matrices. For example, cubic spline can not be used with nan-including arrays, ```python In [1]: x = np.arange(4) ...: y = np.array([0, 1, np.nan, 3]) ...: In [2]: # 'linear' works as expected ...: interp1d(x, y, kind='linear')([0.5, 1.5]) ...: Out[2]: array([0.5, nan]) In [3]: # 'cubic' returns an all nan array ...: interp1d(x, y, kind='cubic')([0.5, 1.5]) ...: Out[3]: array([nan, nan]) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386834171,https://api.github.com/repos/pydata/xarray/issues/2104,386834171,MDEyOklzc3VlQ29tbWVudDM4NjgzNDE3MQ==,1217238,2018-05-05T20:52:57Z,2018-05-05T20:52:57Z,MEMBER,"I still need to look at the code here, but I don't think we need separate code paths for NaN vs no-NaNs. For example, suppose we have a DataArray with a coordinate `x=[0, 1, 2]`, then `da.interp(x=0.5, method='linear')` should be equal to `0.5 * (da.sel(x=0) + da.sel(x=1))`, regardless of the values of the DataArray. If either `da[0]` or `da[1]` is NaN, then the result would be NaN, too.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386796193,https://api.github.com/repos/pydata/xarray/issues/2104,386796193,MDEyOklzc3VlQ29tbWVudDM4Njc5NjE5Mw==,6815844,2018-05-05T10:37:12Z,2018-05-05T10:37:12Z,MEMBER,"Maybe the possible API would be ```python da.interp(x=[0, 1, 2], skipna=True) ``` for nan-skipping interpolation and ```python da.interp(x=[0, 1, 2], skipna=False) ``` for faster interpolation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386744768,https://api.github.com/repos/pydata/xarray/issues/2104,386744768,MDEyOklzc3VlQ29tbWVudDM4Njc0NDc2OA==,6815844,2018-05-04T21:53:05Z,2018-05-04T21:53:05Z,MEMBER,"@jhamman, > I just gave your implementation a quick skim and I think there is currently some duplication of efforts so maybe we look there to see what can be combined. Agreed. I will try to generalize your interpolator-adaptor more. @shoyer > perhaps we should give it a shorter name. Maybe interp? Agreed. `interp` sounds nice. > Which NaN are you referring to? > 1. data on the arrays being indexed > 2. Coordinates on the arrays being indexed > 3. Points at which to interpolate I meant 1. `Scipy.interpolate.interp1d` does not support arrays with NaN. So if we want to support this, we need to interpolate the array line by line as `interpolate_na` does, i.e. no vectorization. For the case of multidimensional interpolation of arrays with NaN, it is equivalent with the interpolation on *non-grided data*. It is possible, but the performance will significantly drop. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386658665,https://api.github.com/repos/pydata/xarray/issues/2104,386658665,MDEyOklzc3VlQ29tbWVudDM4NjY1ODY2NQ==,1217238,2018-05-04T16:40:03Z,2018-05-04T16:40:03Z,MEMBER,"First of all, this is awesome! One question: since I think this new interpolation method will be used quite frequently (more often than `interpolate_na` and `reindex`), perhaps we should give it a shorter name. Maybe `interp`? We should also consider adding `interp_like` or `interpolate_like` by analogy to `reindex_like`. > I would like to this method working similar to isel, which may support vectorized interpolation. Yes, I agree. Interpolation should be vectorized similarly to `isel`/`sel`. In particular, in the long term I think we should aim to make `interpolate_at(..., method='nearest')` and `reindex(..., method='nearest')` equivalent. > How many interpolate methods should we support? I think this is fine to start (we can always add more later). > How do we handle nan? Which NaN are you referring to? 1. `data` on the arrays being indexed 2. Coordinates on the arrays being indexed 3. Points at which to interpolate Case (1) should definitely be supported. Especially if data is stored in dask arrays, we cannot necessarily check if there are NaNs. Cases (2) and (3) are not important, because there are relatively few use-cases for NaN in coordinate arrays. > Do we support interpolation along dimension without coordinate? In that case, do we attach new coordinate to the object? Currently we don't support this for `reindex()`, but I suppose we could do so unambiguous. > How should we do if new coordinate has the dimensional coordinate for the dimension to be interpolated? I think the new coordinates should take priority, and the dimension coordinates on the new coordinate should be dropped. This is similar to what we do for `sel()`: ``` In [10]: da = xr.DataArray([0, 0.1, 0.2, 0.1], dims='x', coords={'x': [0, 1, 2, 3]}) ...: In [11]: da.sel(x=xr.DataArray([1, 2], dims=['x'], coords={'x': [1, 3]})) ...: Out[11]: array([0.1, 0.2]) Coordinates: * x (x) int64 1 2 ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386624429,https://api.github.com/repos/pydata/xarray/issues/2104,386624429,MDEyOklzc3VlQ29tbWVudDM4NjYyNDQyOQ==,2443309,2018-05-04T14:46:11Z,2018-05-04T14:46:11Z,MEMBER,"@fujiisoup - This is great. I wonder if we can draw more from the interpolator adapters I built in `missing.py`: https://github.com/pydata/xarray/blob/b9f40cc1da9c45b3dd33a3434b69c3d8fce57138/xarray/core/missing.py#L284-L323 I just gave your implementation a quick skim and I think there is currently some duplication of efforts so maybe we look there to see what can be combined. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317 https://github.com/pydata/xarray/pull/2104#issuecomment-386606569,https://api.github.com/repos/pydata/xarray/issues/2104,386606569,MDEyOklzc3VlQ29tbWVudDM4NjYwNjU2OQ==,10050469,2018-05-04T13:47:15Z,2018-05-04T13:47:15Z,MEMBER,"@fujiisoup this is awesome! So glad you are looking into this. To point 1: ``How many interpolate methods should we support?``. I think that your implementation is a very good first step. From the geosciences point of view I think that [Spline interpolation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.RectBivariateSpline.html#scipy.interpolate.RectBivariateSpline) would be a very good candidate too, but it only works in the 2-dimensional case which makes it a no-go for the general case... ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,320275317