I think netcdftime should have a version of num2date that always returns netcdftime.datetime objects. It's really error prone to functions that turn different types dependent on input values.

On Tue, Mar 20, 2018 at 11:39 AM Spencer Clark notifications@github.com wrote:

It occurred to me that netcdftime.num2date can return datetime.datetime objects in certain circumstances (not just objects that subclass netcdftime.datetime). 5e1c4a8 https://github.com/pydata/xarray/commit/5e1c4a891c22f3c6b54d31ece8c29306483b36a0 provides fixes to allow things to work with datetime.datetime objects as well as more test coverage.

One thing that makes the logic a bit messier here is that determining the calendar type for encoding datetime.datetime objects for faithful roundtripping https://github.com/spencerkclark/xarray/blob/257f08607c3b0cb975a5114948d2f95f941543db/xarray/coding/times.py#L248-L268 is a bit complicated. As far as I can tell, there are two scenarios, each with their own branches of logic:

1. Arrays where the minimum date is before the start of the Gregorian calendar (1582-10-15)
   • If the dates have the type datetime.datetime we encode them using the 'gregorian' calendar.
   • If the dates have the type netcdftime.DatetimeGregorian we encode them using the 'standard' calendar type.
2. Arrays where the minimum date is on or after the start of the Gregorian calendar (1582-10-15)
   • In this scenario, I'm not sure if it is possible to get num2date to return objects of type netcdftime.DatetimeGregorian. Therefore, regardless of whether the dates are of type datetime.datetime or netcdftime.DatetimeGregorian, we encode them using the 'standard' calendar type.

The rationale behind this more complicated logic has to do with how it appears netcdftime.num2date behaves. Here are some minimal examples:

from netcdftime import date2num, num2date, DatetimeGregorian>>> from datetime import datetime units = 'days since 1582-01-01'>>> # Case with minimum date before start of Gregorian calendar>>> arr = date2num([datetime(1582, 10, 1), datetime(1582, 10, 15)], units=units)>>> # To recover datetime.datetime objects, we need to use calendar='gregorian'>>> num2date(arr, units=units, calendar='gregorian') array([datetime.datetime(1582, 10, 1, 0, 0), datetime.datetime(1582, 10, 15, 0, 0)], dtype=object)>>> num2date(arr, units=units, calendar='standard')>>> # Using calendar='standard' results in netcdftime.DatetimeGregorian objects array([netcdftime._netcdftime.DatetimeGregorian(1582, 10, 1, 0, 0, 0, 0, -1, 1), netcdftime._netcdftime.DatetimeGregorian(1582, 10, 15, 0, 0, 0, 0, -1, 1)], dtype=object)

Case with minimum date after start of Gregorian calendar>>> arr = date2num([datetime(1582, 10, 15), datetime(1582, 10, 16)], units=units)>>> # Regardless of the calendar type used for decoding, we get datetime.datetime objects>>> num2date(arr, units=units, calendar='gregorian')

array([datetime.datetime(1582, 10, 15, 0, 0), datetime.datetime(1582, 10, 16, 0, 0)], dtype=object)>>> num2date(arr, units=units, calendar='standard') array([datetime.datetime(1582, 10, 15, 0, 0), datetime.datetime(1582, 10, 16, 0, 0)], dtype=object)

Given the fact that it does not seem possible to decode times after the start of the Gregorian calendar into dates of type netcdftime.DatetimeGregorian it is surprising that test_roundtrip_netcdftime_datetime_data_post_gregorian https://github.com/spencerkclark/xarray/blob/257f08607c3b0cb975a5114948d2f95f941543db/xarray/tests/test_backends.py#L383-L408 passes for DatetimeGregorian objects. It turns out it works, because timedelta arithmetic is valid between DatetimeGregorian and datetime.datetime objects for dates after the start of the Gregorian calendar:

DatetimeGregorian(1, 1, 1) - datetime(1, 1, 1) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "netcdftime/_netcdftime.pyx", line 1624, in netcdftime._netcdftime.datetime.__sub__ValueError: cannot compute the time difference between dates with different calendars>>> DatetimeGregorian(1582, 10, 30) - datetime(1582, 10, 16) datetime.timedelta(14)

(though maybe I should add an assert statement to check the date types, which would cause it to fail as currently constructed).

@jswhit https://github.com/jswhit, @jhamman https://github.com/jhamman -- is this behavior expected for netcdftime? Is there a recommended robust way for determining the calendar type for roundtripping between date2num and num2date, or is what I have in times.py#L248-L268 https://github.com/spencerkclark/xarray/blob/257f08607c3b0cb975a5114948d2f95f941543db/xarray/coding/times.py#L248-L268 sufficient?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/1252#issuecomment-374711420, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1vmkPkfyIf8NHPw_RLJxC_L8DlEEks5tgUz9gaJpZM4L3tsR .