html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/121#issuecomment-42890839,https://api.github.com/repos/pydata/xarray/issues/121,42890839,MDEyOklzc3VlQ29tbWVudDQyODkwODM5,2443309,2014-05-12T21:28:44Z,2014-05-12T21:28:44Z,MEMBER,"Ok, I just spent a few minutes working through a possible (although not ideal) solution for this. It works although it is a bit ugly and quite a bit slower than the standard calendar option. This option returns a `numpy` array with a dtype: `datetime64[ns]`. The downside is that it uses a `datetime` object as an intermediary.
``` python
In [1]:
import pandas as pd
from netCDF4 import num2date, date2num
import datetime
import numpy as np
from xray.conventions import decode_cf_datetime as decode
units = 'days since 0001-01-01'
# pandas time range
times = pd.date_range('2001-01-01-00', end='2001-06-30-23', freq='H')
# numpy array of numeric dates on noleap calendar
noleap_time = date2num(times.to_pydatetime(), units, calendar='noleap')
# numpy array of numeric dates on standard calendar
std_time = date2num(times.to_pydatetime(), units, calendar='standard')
# decoding function using datetime intermediary
def nctime_to_nptime(times):
new = np.empty(len(times), dtype='M8[ns]')
for i, t in enumerate(times):
new[i] = np.datetime64(datetime.datetime(*t.timetuple()[:6]))
return new
In [2]:
# decode noleap_time
%timeit nctime_to_nptime(decode(noleap_time, units, calendar='noleap'))
noleap_datetimes = nctime_to_nptime(num2date(noleap_time, units, calendar='noleap'))
print 'dtype:', noleap_datetimes.dtype
print noleap_datetimes
10 loops, best of 3: 38.8 ms per loop
dtype: datetime64[ns]
['2000-12-31T16:00:00.000000000-0800' '2000-12-31T17:00:00.000000000-0800'
'2000-12-31T18:00:00.000000000-0800' ...,
'2001-06-30T14:00:00.000000000-0700' '2001-06-30T15:00:00.000000000-0700'
'2001-06-30T16:00:00.000000000-0700']
In [3]:
# decode std_time using vectorized converter
%timeit decode(std_time, units, calendar='standard')
standard_datetimes = decode(std_time, units, calendar='standard')
print 'dtype:', standard_datetimes.dtype
print standard_datetimes
1000 loops, best of 3: 243 µs per loop
dtype: datetime64[ns]
['2000-12-31T16:00:00.000000000-0800' '2000-12-31T16:59:59.000000000-0800'
'2000-12-31T17:59:59.000000000-0800' ...,
'2001-06-30T13:59:59.000000000-0700' '2001-06-30T14:59:59.000000000-0700'
'2001-06-30T15:59:59.000000000-0700']
```
Two three things to notice here:
- the `decode_cf_datetime` is having a precision issue on the seconds when using the numpy timedelta approach (used by standard calendars).
- The new function is substantially slower than the numpy vectorization approach (can't get around that though)
- the `dtype` of the returned array matches regardless of the calendar
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,33272937
https://github.com/pydata/xarray/issues/121#issuecomment-42793512,https://api.github.com/repos/pydata/xarray/issues/121,42793512,MDEyOklzc3VlQ29tbWVudDQyNzkzNTEy,2443309,2014-05-12T03:25:35Z,2014-05-12T03:25:35Z,MEMBER,"I think the simplest option would be to develop a function to cast the `netCDF4.datetime` objects into an array of `numpy.datetime64` objects.
Does `xray` do a lot of timedelta operations? If not, I'd say its probably best to live with (for now) the discrepancies between the calendars. Just be sure that you always use the `netCDF4.date2num` when going back to ordinal times.
I've run into issues like this repeatedly and I think it would be really nice if the `datetime` module was calendar aware -- probably not going to happen. The closest thing I've found to this is the `netCDF4 netcdftime.py` which is written in pure python but could be expanded to handle the issues we're talking about.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,33272937