html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/3764#issuecomment-598568339,https://api.github.com/repos/pydata/xarray/issues/3764,598568339,MDEyOklzc3VlQ29tbWVudDU5ODU2ODMzOQ==,1217238,2020-03-13T06:14:37Z,2020-03-13T06:14:37Z,MEMBER,I'm going to merge this to see if it fixes our doc build...,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-598567039,https://api.github.com/repos/pydata/xarray/issues/3764,598567039,MDEyOklzc3VlQ29tbWVudDU5ODU2NzAzOQ==,1217238,2020-03-13T06:09:01Z,2020-03-13T06:09:01Z,MEMBER,It's also worth noting that this note appears to be causing our doc builds to fail: https://readthedocs.org/projects/xray/builds/10604464/,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-598566923,https://api.github.com/repos/pydata/xarray/issues/3764,598566923,MDEyOklzc3VlQ29tbWVudDU5ODU2NjkyMw==,1217238,2020-03-13T06:08:34Z,2020-03-13T06:08:34Z,MEMBER,These changes look good to me. It's definitely not ideal to be overriding all these details on CFTimeIndex -- this why xarray needs its own indexing API -- but for now it seems like about the best we can do.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-596373136,https://api.github.com/repos/pydata/xarray/issues/3764,596373136,MDEyOklzc3VlQ29tbWVudDU5NjM3MzEzNg==,24736507,2020-03-09T07:36:39Z,2020-03-09T07:37:47Z,NONE,"Hello @spencerkclark! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found: There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: ##### Comment last updated at 2020-03-09 07:37:46 UTC","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-592327684,https://api.github.com/repos/pydata/xarray/issues/3764,592327684,MDEyOklzc3VlQ29tbWVudDU5MjMyNzY4NA==,1217238,2020-02-28T05:57:21Z,2020-02-28T05:57:21Z,MEMBER,"+1 for marking as xfail. There’s no point in keeping the master branch failing if we know what the issue is. On Thu, Feb 27, 2020 at 7:22 PM Maximilian Roos wrote: > Re the specific issue, that's a tough one. It could be worth floating on > pandas and seeing if they have thoughts... > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , > or unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-592292603,https://api.github.com/repos/pydata/xarray/issues/3764,592292603,MDEyOklzc3VlQ29tbWVudDU5MjI5MjYwMw==,5635139,2020-02-28T03:22:01Z,2020-02-28T03:22:01Z,MEMBER,"Re the specific issue, that's a tough one. It could be worth floating on pandas and seeing if they have thoughts...","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-592292231,https://api.github.com/repos/pydata/xarray/issues/3764,592292231,MDEyOklzc3VlQ29tbWVudDU5MjI5MjIzMQ==,5635139,2020-02-28T03:20:35Z,2020-02-28T03:20:35Z,MEMBER,"Re master failing, what do we think about `xfail`-ing those tests on master and then adding a PR which undoes the xfail and attempts fixes the issue? I'm cognizant that every PR now fails, which less friendly to contributors. The project home page also states ![image](https://user-images.githubusercontent.com/5635139/75507496-429dad00-59af-11ea-84d5-ab7cc6f70d12.png). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-590476983,https://api.github.com/repos/pydata/xarray/issues/3764,590476983,MDEyOklzc3VlQ29tbWVudDU5MDQ3Njk4Mw==,2448579,2020-02-24T18:22:29Z,2020-02-24T18:22:29Z,MEMBER,ping @shoyer for input.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-589963580,https://api.github.com/repos/pydata/xarray/issues/3764,589963580,MDEyOklzc3VlQ29tbWVudDU4OTk2MzU4MA==,6628425,2020-02-22T14:55:00Z,2020-02-22T14:55:00Z,MEMBER,"I added some updates to this PR this morning that in principle would solve the indexing with `method=""nearest""` from within xarray. Unfortunately though, due to the issue I described in https://github.com/pydata/xarray/pull/3764#issuecomment-586597512, I was not able to come up with a solution that did not require overriding the pandas implementation of `_get_nearest_indexer`. If this seems unacceptable, maybe we can think harder about how we might address this upstream instead (e.g. I think special-casing the change made in https://github.com/pandas-dev/pandas/pull/31511 to just DatetimeIndexes, and preserving the old behavior for everything else, could be sufficient). In addition, for the time being I xfailed `test_indexing_in_series_getitem`, as I think there is agreement that that would be best addressed upstream, https://github.com/pydata/xarray/issues/3751#issuecomment-587572443. Finally in the process of doing this, I cleaned up the implementation of `CFTimeIndex.__sub__`, and added some more test coverage; hopefully now it's a little clearer what cases it's supposed to work for and what cases it is not.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971 https://github.com/pydata/xarray/pull/3764#issuecomment-586597512,https://api.github.com/repos/pydata/xarray/issues/3764,586597512,MDEyOklzc3VlQ29tbWVudDU4NjU5NzUxMg==,6628425,2020-02-15T14:51:05Z,2020-02-15T14:51:05Z,MEMBER,"This is a little bit trickier than I originally anticipated. For indexing with dates very distant from the dates in the index, I'm still running into an issue at this step in `pandas.Index._get_nearest_indexer`: ``` left_distances = np.abs(self[left_indexer] - target) ``` Consider the following the example: ``` In [1]: import numpy as np; import pandas as pd; import xarray as xr In [2]: import cftime In [3]: times = xr.cftime_range('0001', periods=4) In [4]: da = xr.DataArray(range(4), coords=[('time', times)]) In [5]: da.sel(time=cftime.DatetimeGregorian(2000, 1, 1), method='nearest') ``` In this example `self[left_indexer]` is `CFTimeIndex([0001-01-04 00:00:00], dtype='object', name='time')`, while `target` is `Index([2000-01-01 00:00:00], dtype='object')`. The distance between these dates is greater than 292 years, so it cannot be represented in a `TimedeltaIndex`. One could argue that we could fall back on using a generic `Index` of dtype object to store the `datetime.timedelta` object produced in this case: ``` In [6]: left_index = xr.CFTimeIndex([cftime.DatetimeGregorian(1, 1, 4)]) In [7]: target = pd.Index([cftime.DatetimeGregorian(2000, 1, 1)]) In [8]: difference = left_index - target In [9]: difference Out[9]: Index([-730118 days, 0:00:00], dtype='object') ``` A problem occurs though when we try to take the absolute value of this index. Pandas (I think reasonably so) tries to detect the datatype of the result and construct a new index. In doing so it tries to create a `TimedeltaIndex`, but cannot, because the `timedelta` inside is too large:
Result of np.abs(difference)

``` In [10]: np.abs(difference) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) ~/Software/pandas/pandas/_libs/tslibs/timedeltas.pyx in pandas._libs.tslibs.timedeltas.array_to_timedelta64() TypeError: Expected unicode, got datetime.timedelta During handling of the above exception, another exception occurred: OverflowError Traceback (most recent call last) in ----> 1 np.abs(difference) ~/Software/pandas/pandas/core/indexes/base.py in __array_wrap__(self, result, context) 628 629 attrs = self._get_attributes_dict() --> 630 return Index(result, **attrs) 631 632 @cache_readonly ~/Software/pandas/pandas/core/indexes/base.py in __new__(cls, data, dtype, copy, name, tupleize_cols, **kwargs) 412 413 if dtype is None: --> 414 new_data, new_dtype = _maybe_cast_data_without_dtype(subarr) 415 if new_dtype is not None: 416 return cls( ~/Software/pandas/pandas/core/indexes/base.py in _maybe_cast_data_without_dtype(subarr) 5711 5712 elif inferred.startswith(""timedelta""): -> 5713 data = TimedeltaArray._from_sequence(subarr, copy=False) 5714 return data, data.dtype 5715 elif inferred == ""period"": ~/Software/pandas/pandas/core/arrays/timedeltas.py in _from_sequence(cls, data, dtype, copy, freq, unit) 212 freq, freq_infer = dtl.maybe_infer_freq(freq) 213 --> 214 data, inferred_freq = sequence_to_td64ns(data, copy=copy, unit=unit) 215 freq, freq_infer = dtl.validate_inferred_freq(freq, inferred_freq, freq_infer) 216 ~/Software/pandas/pandas/core/arrays/timedeltas.py in sequence_to_td64ns(data, copy, unit, errors) 938 if is_object_dtype(data.dtype) or is_string_dtype(data.dtype): 939 # no need to make a copy, need to convert if string-dtyped --> 940 data = objects_to_td64ns(data, unit=unit, errors=errors) 941 copy = False 942 ~/Software/pandas/pandas/core/arrays/timedeltas.py in objects_to_td64ns(data, unit, errors) 1047 values = np.array(data, dtype=np.object_, copy=False) 1048 -> 1049 result = array_to_timedelta64(values, unit=unit, errors=errors) 1050 return result.view(""timedelta64[ns]"") 1051 ~/Software/pandas/pandas/_libs/tslibs/timedeltas.pyx in pandas._libs.tslibs.timedeltas.array_to_timedelta64() ~/Software/pandas/pandas/_libs/tslibs/timedeltas.pyx in pandas._libs.tslibs.timedeltas.convert_to_timedelta64() ~/Software/pandas/pandas/_libs/tslibs/timedeltas.pyx in pandas._libs.tslibs.timedeltas.delta_to_nanoseconds() OverflowError: Python int too large to convert to C long ```

So I'm a little bit back to the drawing board regarding a solution for this. Part of me is tempted to write an overriding version of `_get_nearest_indexer` for `CFTimeIndex` that basically restores the old behavior; I'm not sure how dangerous that would be to rely on though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,563202971