html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/3596#issuecomment-602033234,https://api.github.com/repos/pydata/xarray/issues/3596,602033234,MDEyOklzc3VlQ29tbWVudDYwMjAzMzIzNA==,12862013,2020-03-21T11:50:23Z,2020-03-21T11:50:44Z,CONTRIBUTOR,"Wow, didn't look for a week. Very happy 🎉! Thank you @dcherian and @max-sixty for all your input.
> Must be one of the largest first contributions...
I was inspired by the hacktoberfest last year. Took a little bit over a month though 😛
","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-601282600,https://api.github.com/repos/pydata/xarray/issues/3596,601282600,MDEyOklzc3VlQ29tbWVudDYwMTI4MjYwMA==,5635139,2020-03-19T16:36:55Z,2020-03-19T16:36:55Z,MEMBER,"Thanks @mark-boer ! Must be one of the largest first contributions...
+1 re merge + experimental warning; maybe we should do this more often. Cheers @dcherian ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-601217610,https://api.github.com/repos/pydata/xarray/issues/3596,601217610,MDEyOklzc3VlQ29tbWVudDYwMTIxNzYxMA==,2448579,2020-03-19T14:41:39Z,2020-03-19T14:41:39Z,MEMBER,"Merging. I've added an experimental warning to the docstrings and we can discuss the IndexVariable situation here: https://github.com/pydata/xarray/issues/3868
Thanks @mark-boer this is a major contribution for your first PR!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-561843034,https://api.github.com/repos/pydata/xarray/issues/3596,561843034,MDEyOklzc3VlQ29tbWVudDU2MTg0MzAzNA==,24736507,2019-12-04T21:18:57Z,2020-03-18T21:58:03Z,NONE,"Hello @mark-boer! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:
There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:
##### Comment last updated at 2020-03-18 21:58:02 UTC","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-598730380,https://api.github.com/repos/pydata/xarray/issues/3596,598730380,MDEyOklzc3VlQ29tbWVudDU5ODczMDM4MA==,5635139,2020-03-13T13:52:13Z,2020-03-13T13:52:13Z,MEMBER,"Agree!
We could add an ""Experimental"" label and then worry less about future changes","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-598717521,https://api.github.com/repos/pydata/xarray/issues/3596,598717521,MDEyOklzc3VlQ29tbWVudDU5ODcxNzUyMQ==,2448579,2020-03-13T13:23:37Z,2020-03-13T13:23:37Z,MEMBER,"Extremely good points @mark-boer
I propose we merge and open an issue to decide what to do with IndexVariables. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-596837204,https://api.github.com/repos/pydata/xarray/issues/3596,596837204,MDEyOklzc3VlQ29tbWVudDU5NjgzNzIwNA==,5635139,2020-03-10T00:08:11Z,2020-03-10T00:08:11Z,MEMBER,"> In some instances extrapolating all coords, can lead to some unwanted behaviour. Would you suggest we only interpolate the indexes?
>
> How would we handle unsorted indexes?
>
> How would we extrapolate all the different kind of indexes, like the MultiIndex or CategoricalIndex?
I agree; I can't see easy solutions to these. If there are evenly spaced indexes (e.g. dates, grids), then it's easy to know what to do. But there are plenty of times it's difficult, if not intractable.
One option to merge something useful without resolving these questions is to return Variables only and label this method experimental. I think this is immediately useful for the cases where these difficult questions don't need to be answered.
Of course if there are good answers to the questions, then even better!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-596645749,https://api.github.com/repos/pydata/xarray/issues/3596,596645749,MDEyOklzc3VlQ29tbWVudDU5NjY0NTc0OQ==,12862013,2020-03-09T16:48:11Z,2020-03-09T16:48:11Z,CONTRIBUTOR,"> :) I think we need to extrapolate indexes by default. It seems like the most sensible option.
Sorry, obviously that is a solution ;-). But I do have some concerns:
In some instances extrapolating all coords, can lead to some unwanted behaviour. Would you suggest we only interpolate the indexes?
How would we handle unsorted indexes?
How would we extrapolate all the different kind of indexes, like the MultiIndex or CategoricalIndex?
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-596363935,https://api.github.com/repos/pydata/xarray/issues/3596,596363935,MDEyOklzc3VlQ29tbWVudDU5NjM2MzkzNQ==,2448579,2020-03-09T07:10:40Z,2020-03-09T07:10:40Z,MEMBER,"> Hmm, I don't really see a solution. What do you suggest?
:) I think we need to extrapolate indexes by default. It seems like the most sensible option.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-596240476,https://api.github.com/repos/pydata/xarray/issues/3596,596240476,MDEyOklzc3VlQ29tbWVudDU5NjI0MDQ3Ng==,12862013,2020-03-08T19:09:17Z,2020-03-08T19:09:17Z,CONTRIBUTOR,"> Edit: But it's more awkward for _indexes_ than non-index coords. The index becomes less useful with non-unique values, and generally indexes don't have nulls. I'm not sure what the other options would be: to some extent it's the intersection of `pad` with xarray's data model.
Hmm, I don't really see a solution. What do you suggest?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-595506745,https://api.github.com/repos/pydata/xarray/issues/3596,595506745,MDEyOklzc3VlQ29tbWVudDU5NTUwNjc0NQ==,5635139,2020-03-06T00:04:03Z,2020-03-06T02:08:02Z,MEMBER,"> I do agree it can be confusing, but it is not unique in xarray. Dataset.shift only shifts `data_vars`, `bfill` and `ffill` only fill `data_vars`, etc.
I agree, I wouldn't have expected coords to be included given existing behavior. People can switch coords <> data_vars as needed, so there's an escape hatch
Edit: But it's more awkward for _indexes_ than non-index coords. The index becomes less useful with non-unique values, and generally indexes don't have nulls. I'm not sure what the other options would be: to some extent it's the intersection of `pad` with xarray's data model. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-595498610,https://api.github.com/repos/pydata/xarray/issues/3596,595498610,MDEyOklzc3VlQ29tbWVudDU5NTQ5ODYxMA==,12862013,2020-03-05T23:34:18Z,2020-03-05T23:34:18Z,CONTRIBUTOR,"> The big outstanding issue is what to do about dimension coordinates or indexes. Currently this PR treats all variables in `coords` different from those in `data_vars`. I think this is confusing.
I do agree it can be confusing, but it is not unique in xarray. Dataset.shift only shifts `data_vars`, `bfill` and `ffill` only fill `data_vars`, etc.
Personally I think that extrapolating data coordinates without specifically settings a keyword or flag could also be confusing. I occasionally have data in my `coords` that make no sense to extrapolate. I do agree that adding an extrapolate_coord option or keyword of some sorts would be cool in the future.
Both defaults could be really useful, I'm still in a bit of doubt. Are there any other people that might want to weigh in?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-595244925,https://api.github.com/repos/pydata/xarray/issues/3596,595244925,MDEyOklzc3VlQ29tbWVudDU5NTI0NDkyNQ==,5635139,2020-03-05T14:02:45Z,2020-03-05T14:02:45Z,MEMBER,"This looks excellent @mark-boer , thank you!
I will try and have a proper look through today (but don't wait for me)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-595202061,https://api.github.com/repos/pydata/xarray/issues/3596,595202061,MDEyOklzc3VlQ29tbWVudDU5NTIwMjA2MQ==,2448579,2020-03-05T12:22:02Z,2020-03-05T12:22:02Z,MEMBER,"I pushed some minor changes.
I think this is ready to go in.
The big outstanding issue is what to do about dimension coordinates or indexes. Currently this PR treats all variables in `coords` different from those in `data_vars`. I think this is confusing.
I am thinking that we want to use linear extrapolation for IndexVariables by default and apply the same padding mode to all other variables. The reasoning being that IndexVariables with NaNs are hard to deal with and it's hard to fill them in: `padded[""x""] = padded.x.drop_vars(""x"").interpolate_na(""x"", fill_value=""extrapolate"")`
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-585612496,https://api.github.com/repos/pydata/xarray/issues/3596,585612496,MDEyOklzc3VlQ29tbWVudDU4NTYxMjQ5Ng==,12862013,2020-02-13T08:37:16Z,2020-02-13T08:37:16Z,CONTRIBUTOR,"Hi @dcherian
Once again an excellent code review :-)
I adressed most points in your review, except for the function signature of `DataArray.pad`. I think I prefer the default values of None. Now numpy/Dask will throw an error when you set invalid combinations e.g. `constant_values=0`, and set the `mode=""mean""`. The np.nan is unfortunate, but this is the difference between np.nan and xarray.dtypes.NaN. I could set the default to `dtypes.NaN`, but then you lose the automatic error when setting an explicit value with `mode!=""constant""`.
I can also see that something like coords_mode being useful, but I think it would be wise to wait and see how pad gets used and what issues we run into along the way.
I also ran into a small issue with pint (https://github.com/hgrecco/pint/issues/1026), but I'll get to that once this PR is merged.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-578549057,https://api.github.com/repos/pydata/xarray/issues/3596,578549057,MDEyOklzc3VlQ29tbWVudDU3ODU0OTA1Nw==,12862013,2020-01-26T22:16:42Z,2020-01-26T22:16:42Z,CONTRIBUTOR,"Hi @dcherian and @fujiisoup, appart from the issues I raised in my last comment, I think this PR is close to done.
Thx in advance","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-570057684,https://api.github.com/repos/pydata/xarray/issues/3596,570057684,MDEyOklzc3VlQ29tbWVudDU3MDA1NzY4NA==,12862013,2020-01-01T14:29:15Z,2020-01-01T14:29:15Z,CONTRIBUTOR,"Hi everyone, happy new year :tada:
I feel like this PR is slowly getting into a descent shape. But I could use your input on a couple of subjects:
**How should we handle the default padding?**
I currently pad with `dtypes.NaN` this is similar behaviour as used by the replaced variable.pad_with_fill_value` and is consistent with the `shift` method.
We could chose to pad with 0's as numpy does it, or we could force the user to choose a constantt_value, but I don't think I would prefer this.
**How should the coordinates of a DataArray/set be padded?**
I chose default padding except for modes ""edge"", ""reflect"", ""symmetric"", ""wrap"". @dcherian noted that this could make sense in some situations, but that I would require some further discussion.
Personally I think it more often than not it makes sense, but padding with NaN's should also work fine.
**dask_array_compat**
Also I created a workaround for the changing of dtype by Dask.pad mode=mean and added an additional check to validate the output shape of dask.pad. Let me know what you think about this, I'm not 100% convinced...
I used a couple of `# type: ignore` as any work around the mypy errors lead to uglier code. But if any of you have some suggestions, I'm happy to hear them","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-564904765,https://api.github.com/repos/pydata/xarray/issues/3596,564904765,MDEyOklzc3VlQ29tbWVudDU2NDkwNDc2NQ==,12862013,2019-12-12T08:35:06Z,2019-12-12T08:35:06Z,CONTRIBUTOR,"> It seems like we have some value mismatches on `dask==1.2`.
The reason that `mean` and `reflect` are marked as xfail are because there too there are value mismatches. I'll see if I can find a workaround for this failing test.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-563311262,https://api.github.com/repos/pydata/xarray/issues/3596,563311262,MDEyOklzc3VlQ29tbWVudDU2MzMxMTI2Mg==,2448579,2019-12-09T16:11:46Z,2019-12-09T16:11:46Z,MEMBER,"It seems like we have some value mismatches on `dask==1.2`. @fujiisoup is this the error you found?
```
=================================== FAILURES ===================================
__________ TestVariableWithDask.test_pad[xr_arg0-np_arg0-linear_ramp] __________
self =
mode = 'linear_ramp', xr_arg = {'x': (2, 1)}, np_arg = ((2, 1), (0, 0), (0, 0))
@pytest.mark.parametrize(
""mode"",
[
pytest.param(""mean"", marks=pytest.mark.xfail),
pytest.param(""median"", marks=pytest.mark.xfail),
pytest.param(""reflect"", marks=pytest.mark.xfail),
""edge"",
""linear_ramp"",
""maximum"",
""minimum"",
""symmetric"",
""wrap"",
],
)
@pytest.mark.parametrize(
""xr_arg, np_arg"",
[
[{""x"": (2, 1)}, ((2, 1), (0, 0), (0, 0))],
[{""y"": (0, 3)}, ((0, 0), (0, 3), (0, 0))],
[{""x"": (3, 1), ""z"": (2, 0)}, ((3, 1), (0, 0), (2, 0))],
],
)
def test_pad(self, mode, xr_arg, np_arg):
data = np.arange(4 * 3 * 2).reshape(4, 3, 2)
v = self.cls([""x"", ""y"", ""z""], data)
actual = v.pad(mode=mode, **xr_arg)
expected = np.pad(data, np_arg, mode=mode,)
> assert_array_equal(actual, expected)
E AssertionError:
E Arrays are not equal
E
E (mismatch 2.3809523809523796%)
E x: array([[[ 0, 0],
E [ 0, 0],
E [ 0, 0]],...
E y: array([[[ 0, 0],
E [ 0, 0],
E [ 0, 0]],...
xarray/tests/test_variable.py:821: AssertionError
__________ TestVariableWithDask.test_pad[xr_arg1-np_arg1-linear_ramp] __________
self =
mode = 'linear_ramp', xr_arg = {'y': (0, 3)}, np_arg = ((0, 0), (0, 3), (0, 0))
@pytest.mark.parametrize(
""mode"",
[
pytest.param(""mean"", marks=pytest.mark.xfail),
pytest.param(""median"", marks=pytest.mark.xfail),
pytest.param(""reflect"", marks=pytest.mark.xfail),
""edge"",
""linear_ramp"",
""maximum"",
""minimum"",
""symmetric"",
""wrap"",
],
)
@pytest.mark.parametrize(
""xr_arg, np_arg"",
[
[{""x"": (2, 1)}, ((2, 1), (0, 0), (0, 0))],
[{""y"": (0, 3)}, ((0, 0), (0, 3), (0, 0))],
[{""x"": (3, 1), ""z"": (2, 0)}, ((3, 1), (0, 0), (2, 0))],
],
)
def test_pad(self, mode, xr_arg, np_arg):
data = np.arange(4 * 3 * 2).reshape(4, 3, 2)
v = self.cls([""x"", ""y"", ""z""], data)
actual = v.pad(mode=mode, **xr_arg)
expected = np.pad(data, np_arg, mode=mode,)
> assert_array_equal(actual, expected)
E AssertionError:
E Arrays are not equal
E
E (mismatch 16.66666666666667%)
E x: array([[[ 0, 1],
E [ 2, 3],
E [ 4, 5],...
E y: array([[[ 0, 1],
E [ 2, 3],
E [ 4, 5],...
xarray/tests/test_variable.py:821: AssertionError
__________ TestVariableWithDask.test_pad[xr_arg2-np_arg2-linear_ramp] __________
self =
mode = 'linear_ramp', xr_arg = {'x': (3, 1), 'z': (2, 0)}
np_arg = ((3, 1), (0, 0), (2, 0))
@pytest.mark.parametrize(
""mode"",
[
pytest.param(""mean"", marks=pytest.mark.xfail),
pytest.param(""median"", marks=pytest.mark.xfail),
pytest.param(""reflect"", marks=pytest.mark.xfail),
""edge"",
""linear_ramp"",
""maximum"",
""minimum"",
""symmetric"",
""wrap"",
],
)
@pytest.mark.parametrize(
""xr_arg, np_arg"",
[
[{""x"": (2, 1)}, ((2, 1), (0, 0), (0, 0))],
[{""y"": (0, 3)}, ((0, 0), (0, 3), (0, 0))],
[{""x"": (3, 1), ""z"": (2, 0)}, ((3, 1), (0, 0), (2, 0))],
],
)
def test_pad(self, mode, xr_arg, np_arg):
data = np.arange(4 * 3 * 2).reshape(4, 3, 2)
v = self.cls([""x"", ""y"", ""z""], data)
actual = v.pad(mode=mode, **xr_arg)
expected = np.pad(data, np_arg, mode=mode,)
> assert_array_equal(actual, expected)
E AssertionError:
E Arrays are not equal
E
E (mismatch 5.208333333333329%)
E x: array([[[ 0, 0, 0, 0],
E [ 0, 0, 0, 0],
E [ 0, 0, 0, 0]],...
E y: array([[[ 0, 0, 0, 0],
E [ 0, 0, 0, 0],
E [ 0, 0, 0, 0]],...
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062
https://github.com/pydata/xarray/pull/3596#issuecomment-562821225,https://api.github.com/repos/pydata/xarray/issues/3596,562821225,MDEyOklzc3VlQ29tbWVudDU2MjgyMTIyNQ==,6815844,2019-12-07T06:47:32Z,2019-12-07T06:47:32Z,MEMBER,"Hi, @mark-boer.
In #3587, I tried using dask's pad method but noticed a few bugs in older (but newer than 1.2) dask.
For me, it would be very welcome if you add this method to dask_array_compat.
Then, I would wait for merging #3587 until this PR is completed.
Thanks for your contribution :)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,532940062