home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

10 rows where issue = 320275317 and user = 10050469 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • fmaussion · 10 ✖

issue 1

  • implement interp() · 10 ✖

author_association 1

  • MEMBER 10
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
388009361 https://github.com/pydata/xarray/pull/2104#issuecomment-388009361 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4ODAwOTM2MQ== fmaussion 10050469 2018-05-10T09:57:26Z 2018-05-10T09:57:26Z MEMBER

Is there any problem if we always keep attrs in the interpolation?

In my opinion it should be the default, yes.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
388009096 https://github.com/pydata/xarray/pull/2104#issuecomment-388009096 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4ODAwOTA5Ng== fmaussion 10050469 2018-05-10T09:56:15Z 2018-05-10T09:56:15Z MEMBER

BTW, do you have any idea about an example for the new interp feature?

A quick shot would be to use the existing sample dataset:

```python import matplotlib.pyplot as plt import xarray as xr import numpy as np

Raw data

ds = xr.tutorial.load_dataset('air_temperature') ds.air.isel(time=0).plot() plt.title('Raw data')

Interpolated data

new_lon = np.linspace(ds.lon[0], ds.lon[-1], ds.dims['lon'] * 4) new_lat = np.linspace(ds.lat[0], ds.lat[-1], ds.dims['lat'] * 4) dsi = ds.interp(lon=new_lon, lat=new_lat) dsi.air.isel(time=0).plot() plt.title('Interpolated data') ``` Which produces the plots:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
387993706 https://github.com/pydata/xarray/pull/2104#issuecomment-387993706 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4Nzk5MzcwNg== fmaussion 10050469 2018-05-10T08:48:12Z 2018-05-10T08:48:12Z MEMBER

As the interpolation routine can also be used for non-uniform gridded data, I don't think passing np.arange(NN) to scipy is a good idea (it will change the result if higher order method such as 'cubic' is used).

Right. Another feature of geospatial grids is that they are regularly spaced, which is not always the case here. Forget about my idea for now.

I tested your updated branch and we now come to the same results with my test data! I took the liberty to edit your comment because in your current implementation assume_sorted is False per default.

Another wish (sorry! ;): it would be great to implement the keep_attrs kwarg as well, because interpolation often conserves the units of data. Thanks!!!!

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
387848141 https://github.com/pydata/xarray/pull/2104#issuecomment-387848141 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4Nzg0ODE0MQ== fmaussion 10050469 2018-05-09T19:24:27Z 2018-05-09T19:24:27Z MEMBER

@fujiisoup I gave a go at your branch this evening, trying a few things on real data cases. First of all: this is great! Really looking forward to see this in xarray.

I noticed a small issue: currently this doesn't work with decreasing coordinates:

python nx = 5 ny = 4 lon = np.arange(nx) lat = np.arange(ny)[::-1] data = np.arange(nx*ny).reshape((ny, nx)) da = xr.DataArray(data, dims=('lat', 'lon'), coords={'lon':lon, 'lat':lat}) da.interp(lon=[1.1, 2.2, 3.3], lat=[1.2, 2.3]) -> ValueError from SciPy: The points in dimension 0 must be strictly ascending Traceback:

--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-87-98b53090c21d> in <module>() ----> 1 da.interp(lon=[1.1, 2.2, 3.3], lat=[2.3, 1.2]) ~/tmp/xarray-fuji/xarray/core/dataarray.py in interp(self, method, kwargs, **coords) 912 """ 913 ds = self._to_temp_dataset().interp( --> 914 method=method, kwargs=kwargs, **coords) 915 return self._from_temp_dataset(ds) 916 ~/tmp/xarray-fuji/xarray/core/dataset.py in interp(self, method, kwargs, **coords) 1824 if name not in [k for k, v in indexers_list]: 1825 variables[name] = missing.interp( -> 1826 var, var_indexers, method, **kwargs) 1827 1828 coord_names = set(variables).intersection(self._coord_names) ~/tmp/xarray-fuji/xarray/core/missing.py in interp(obj, indexes_coords, method, **kwargs) 441 kwargs={'x': x, 'new_x': destination, 'method': method, 442 'kwargs': kwargs}, --> 443 keep_attrs=True) 444 445 if all(x1.dims == new_x1.dims for x1, new_x1 in zip(x, new_x)): ~/tmp/xarray-fuji/xarray/core/computation.py in apply_ufunc(func, *args, **kwargs) 934 keep_attrs=keep_attrs) 935 elif any(isinstance(a, Variable) for a in args): --> 936 return variables_ufunc(*args) 937 else: 938 return apply_array_ufunc(func, *args, dask=dask) ~/tmp/xarray-fuji/xarray/core/computation.py in apply_variable_ufunc(func, *args, **kwargs) 563 raise ValueError('unknown setting for dask array handling in ' 564 'apply_ufunc: {}'.format(dask)) --> 565 result_data = func(*input_data) 566 567 if signature.num_outputs > 1: ~/tmp/xarray-fuji/xarray/core/missing.py in interp_func(obj, x, new_x, method, kwargs) 499 500 func, kwargs = _get_interpolator_nd(method, **kwargs) --> 501 return _interpnd(obj, x, new_x, func, kwargs) 502 503 ~/tmp/xarray-fuji/xarray/core/missing.py in _interpnd(obj, x, new_x, func, kwargs) 518 # stack new_x to 1 vector, with reshape 519 xi = np.stack([x1.values.ravel() for x1 in new_x], axis=-1) --> 520 rslt = func(x, obj, xi, **kwargs) 521 # move back the interpolation axes to the last position 522 rslt = rslt.transpose(range(-rslt.ndim + 1, 1)) ~/.pyvirtualenvs/py3/lib/python3.5/site-packages/scipy/interpolate/interpolate.py in interpn(points, values, xi, method, bounds_error, fill_value) 2589 if not np.all(np.diff(p) > 0.): 2590 raise ValueError("The points in dimension %d must be strictly " -> 2591 "ascending" % i) 2592 if not np.asarray(p).ndim == 1: 2593 raise ValueError("The points in dimension %d must be " ValueError: The points in dimension 0 must be strictly ascending

Decreasing coordinates are really frequent in climate data (latitudes are often decreasing, don't ask me why), so I wondered how I implemented this in salem, which brings me to the second point: I think we need an interp() equivalent to isel() too, which is the one I could make use of with salem (and, maybe, several other georeferencing tools).

The way salem works is that it uses the concept of geospatial grid where the points to interpolate are defined in a local coordinate system which is always positive and increasing. That is, if I want to use xarray's interpolation routines, I'd like to be able to tell xarray that I'd like to interpolate at "indices [1.1, 2.1, 3.1]" for example, where these can really be understood as indexes in the way da.isel() sees them.

I don't know if we really need a i_interp for this though, or if it could be solved with a kwarg to the existing API. The implementation for you would be straightforward, you'd just have to pass np.arange(NN) to scipy instead of the actual coordinates.

Thoughts?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
387634881 https://github.com/pydata/xarray/pull/2104#issuecomment-387634881 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4NzYzNDg4MQ== fmaussion 10050469 2018-05-09T06:33:26Z 2018-05-09T06:33:26Z MEMBER

I prefer not to support this now for keeping the interpolation rule simple.

Agreed. Users can turn their 1D non dimension coordinates into dimension coordinates if they want to.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
387387039 https://github.com/pydata/xarray/pull/2104#issuecomment-387387039 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4NzM4NzAzOQ== fmaussion 10050469 2018-05-08T12:33:36Z 2018-05-08T12:33:36Z MEMBER

Let us know when this is ready for review! I'm happy to have a look at it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
387386033 https://github.com/pydata/xarray/pull/2104#issuecomment-387386033 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4NzM4NjAzMw== fmaussion 10050469 2018-05-08T12:29:53Z 2018-05-08T12:29:53Z MEMBER

Honestly, I am not very happy with kwargs={} arguments (it would be more beautiful if they can be keyword arguments), but I think it is a necessary pain

I don't think it's that bad. It makes it clear that the arguments are passed over to the underlying function, and that xarray is not doing all the magic by itself.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
386876126 https://github.com/pydata/xarray/pull/2104#issuecomment-386876126 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4Njg3NjEyNg== fmaussion 10050469 2018-05-06T12:31:27Z 2018-05-06T12:31:27Z MEMBER

I think the addition of the linear interpolation is still nice, but it would be nicer if we could also implement nan-skipping interpolations, which will enable us to use higher order interpolation with nan-including arrays.

Do you mean you want to use unstructured data interpolation routines? (https://docs.scipy.org/doc/scipy/reference/interpolate.html#multivariate-interpolation)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
386869104 https://github.com/pydata/xarray/pull/2104#issuecomment-386869104 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4Njg2OTEwNA== fmaussion 10050469 2018-05-06T10:24:00Z 2018-05-06T10:24:00Z MEMBER

linear and nearest can handle NaN intuitively, but other methods need to invert some matrices

Why don't leave the NaN handling part to the actual function doing the interpolation? RegularGridInterpolator handles NaNs pretty much the same way @shoyer describes it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317
386606569 https://github.com/pydata/xarray/pull/2104#issuecomment-386606569 https://api.github.com/repos/pydata/xarray/issues/2104 MDEyOklzc3VlQ29tbWVudDM4NjYwNjU2OQ== fmaussion 10050469 2018-05-04T13:47:15Z 2018-05-04T13:47:15Z MEMBER

@fujiisoup this is awesome! So glad you are looking into this.

To point 1: How many interpolate methods should we support?.

I think that your implementation is a very good first step. From the geosciences point of view I think that Spline interpolation would be a very good candidate too, but it only works in the 2-dimensional case which makes it a no-go for the general case...

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  implement interp() 320275317

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1200.01ms · About: xarray-datasette