home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 1209664972

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6191#issuecomment-1209664972 https://api.github.com/repos/pydata/xarray/issues/6191 1209664972 IC_kwDOAMm_X85IGgXM 868027 2022-08-09T17:30:07Z 2022-08-09T17:30:07Z CONTRIBUTOR

Some additional info for when how to figure out the best way to address this.

For the decode using pandas approach, two things I tried worked: using a pandas.array with a nullable integer data type, or simulating what happens on x86_64 systems by checking for nans in the incoming array and setting those positions to numpy.iinfo(np.int64).min.

the pandas nullable integer array: ```python

# note that is a capital i Int64 to use the nullable type.
flat_num_dates_ns_int = pd.array(flat_num_dates * _NS_PER_TIME_DELTA[delta], dtype="Int64")

simulate x86:python

flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(
    np.int64
)

flat_num_dates_ns_int[np.isnan(flat_num_dates)] = np.iinfo(np.int64).min

```

The pandas solution is explicitly experimental in their docs, and the emulate version just feels "hacky" to me. These don't break any existing tests on my local machine.

cftime itself has no support for nan type missing values and will fail:

(on x86_64) ```python

import numpy as np from xarray.coding.times import decode_cf_datetime decode_cf_datetime(np.array([0, np.nan]), "days since 1950-01-01", use_cftime=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/abarna/.pyenv/versions/3.8.5/lib/python3.8/site-packages/xarray/coding/times.py", line 248, in decode_cf_datetime dates = _decode_datetime_with_cftime(flat_num_dates, units, calendar) File "/home/abarna/.pyenv/versions/3.8.5/lib/python3.8/site-packages/xarray/coding/times.py", line 164, in _decode_datetime_with_cftime cftime.num2date(num_dates, units, calendar, only_use_cftime_datetimes=True) File "src/cftime/_cftime.pyx", line 484, in cftime._cftime.num2date TypeError: unsupported operand type(s) for +: 'cftime._cftime.DatetimeGregorian' and 'NoneType' ```

cftime is happy with masked arrays: ```python

import cftime a1 = np.ma.masked_invalid(np.array([0, np.nan])) cftime.num2date(a1, "days since 1950-01-01") masked_array(data=[cftime.DatetimeGregorian(1950, 1, 1, 0, 0, 0, 0), --], mask=[False, True], fill_value='?', dtype=object) ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1114351614
Powered by Datasette · Queries took 0.644ms · About: xarray-datasette