home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 42890839

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/121#issuecomment-42890839 https://api.github.com/repos/pydata/xarray/issues/121 42890839 MDEyOklzc3VlQ29tbWVudDQyODkwODM5 2443309 2014-05-12T21:28:44Z 2014-05-12T21:28:44Z MEMBER

Ok, I just spent a few minutes working through a possible (although not ideal) solution for this. It works although it is a bit ugly and quite a bit slower than the standard calendar option. This option returns a numpy array with a dtype: datetime64[ns]. The downside is that it uses a datetime object as an intermediary.

``` python In [1]: import pandas as pd from netCDF4 import num2date, date2num import datetime import numpy as np from xray.conventions import decode_cf_datetime as decode

units = 'days since 0001-01-01'

pandas time range

times = pd.date_range('2001-01-01-00', end='2001-06-30-23', freq='H')

numpy array of numeric dates on noleap calendar

noleap_time = date2num(times.to_pydatetime(), units, calendar='noleap')

numpy array of numeric dates on standard calendar

std_time = date2num(times.to_pydatetime(), units, calendar='standard')

decoding function using datetime intermediary

def nctime_to_nptime(times): new = np.empty(len(times), dtype='M8[ns]') for i, t in enumerate(times): new[i] = np.datetime64(datetime.datetime(*t.timetuple()[:6])) return new

In [2]:

decode noleap_time

%timeit nctime_to_nptime(decode(noleap_time, units, calendar='noleap')) noleap_datetimes = nctime_to_nptime(num2date(noleap_time, units, calendar='noleap')) print 'dtype:', noleap_datetimes.dtype print noleap_datetimes 10 loops, best of 3: 38.8 ms per loop dtype: datetime64[ns] ['2000-12-31T16:00:00.000000000-0800' '2000-12-31T17:00:00.000000000-0800' '2000-12-31T18:00:00.000000000-0800' ..., '2001-06-30T14:00:00.000000000-0700' '2001-06-30T15:00:00.000000000-0700' '2001-06-30T16:00:00.000000000-0700']

In [3]:

decode std_time using vectorized converter

%timeit decode(std_time, units, calendar='standard') standard_datetimes = decode(std_time, units, calendar='standard') print 'dtype:', standard_datetimes.dtype print standard_datetimes 1000 loops, best of 3: 243 µs per loop dtype: datetime64[ns] ['2000-12-31T16:00:00.000000000-0800' '2000-12-31T16:59:59.000000000-0800' '2000-12-31T17:59:59.000000000-0800' ..., '2001-06-30T13:59:59.000000000-0700' '2001-06-30T14:59:59.000000000-0700' '2001-06-30T15:59:59.000000000-0700'] ```

Two three things to notice here: - the decode_cf_datetime is having a precision issue on the seconds when using the numpy timedelta approach (used by standard calendars). - The new function is substantially slower than the numpy vectorization approach (can't get around that though) - the dtype of the returned array matches regardless of the calendar

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  33272937
Powered by Datasette · Queries took 1.004ms · About: xarray-datasette