home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

1 row where author_association = "MEMBER", issue = 374070147 and user = 6628425 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • spencerkclark · 1 ✖

issue 1

  • to_netcdf() fails because of datetime encoding · 1 ✖

author_association 1

  • MEMBER · 1 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
433194914 https://github.com/pydata/xarray/issues/2512#issuecomment-433194914 https://api.github.com/repos/pydata/xarray/issues/2512 MDEyOklzc3VlQ29tbWVudDQzMzE5NDkxNA== spencerkclark 6628425 2018-10-25T20:30:49Z 2018-10-25T20:30:49Z MEMBER

I think this is a bug. The error message in particular makes things confusing. The problem stems from this line, which results in an array of dtype object (see the explanation below): python dts = np.array([dt + timedelta(days=x) for x in range(10)]) As a temporary workaround, if you keep dts as a list before passing it to the Dataset constructor, things will work as you expect: python dts = [dt + timedelta(days=x) for x in range(10)]


By default NumPy will cast a list of datetime.datetime objects to an array with dtype object: ``` In [4]: dts = np.array([dt + timedelta(days=x) for x in range(10)])

In [5]: dts Out[5]: array([datetime.datetime(1999, 1, 1, 0, 0), datetime.datetime(1999, 1, 2, 0, 0), datetime.datetime(1999, 1, 3, 0, 0), datetime.datetime(1999, 1, 4, 0, 0), datetime.datetime(1999, 1, 5, 0, 0), datetime.datetime(1999, 1, 6, 0, 0), datetime.datetime(1999, 1, 7, 0, 0), datetime.datetime(1999, 1, 8, 0, 0), datetime.datetime(1999, 1, 9, 0, 0), datetime.datetime(1999, 1, 10, 0, 0)], dtype=object) If you specify an array of dtype object as a coordinate, xarray currently has some logic that requires that it be cast to a generic `pandas.Index` with dtype object (and will preserve the `datetime.datetime` elements of the array). In [14]: da = xarray.DataArray(range(10), coords=[dts], dims=['time'])

In [15]: da Out[15]: <xarray.DataArray (time: 10)> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) Coordinates: * time (time) object 1999-01-01 1999-01-02 1999-01-03 1999-01-04 ...

In [16]: da.indexes['time'] Out[16]: Index([1999-01-01 00:00:00, 1999-01-02 00:00:00, 1999-01-03 00:00:00, 1999-01-04 00:00:00, 1999-01-05 00:00:00, 1999-01-06 00:00:00, 1999-01-07 00:00:00, 1999-01-08 00:00:00, 1999-01-09 00:00:00, 1999-01-10 00:00:00], dtype='object', name='time') The code where this happens is here: https://github.com/pydata/xarray/blob/b2a377f8dd215a0e1d3cbab61e6ae930347f4063/xarray/core/utils.py#L69-L73 In practice, in the event that an object type array contains `datetime.datetime` objects, this should probably convert them to `np.datetime64` instead and return a `pandas.DatetimeIndex`. This would be more consistent with how xarray currently handles passing an object array of `datetime.datetime` objects as the data argument to a DataArray; there it converts things automatically: In [18]: xarray.DataArray(dts) Out[18]: <xarray.DataArray (dim_0: 10)> array(['1999-01-01T00:00:00.000000000', '1999-01-02T00:00:00.000000000', '1999-01-03T00:00:00.000000000', '1999-01-04T00:00:00.000000000', '1999-01-05T00:00:00.000000000', '1999-01-06T00:00:00.000000000', '1999-01-07T00:00:00.000000000', '1999-01-08T00:00:00.000000000', '1999-01-09T00:00:00.000000000', '1999-01-10T00:00:00.000000000'], dtype='datetime64[ns]') Dimensions without coordinates: dim_0 `` Xarray's logic to encode dates (i.e. what is used when saving datetime data to files) requires that dates are either of typenp.datetime64orcftime.datetime(datetime.datetimeobjects are not supported there). Ifdatetime.datetimearrays were automatically converted tonp.datetime64` arrays in all cases then this would not be an issue.

The error message actually results from the dates being converted to np.datetime64 after the datetime encoding logic is skipped (it happens in the Variable constructor here) and netCDF4 doesn't know how to deal with that data type, but I think the fundamental issue relates to the lines of code I referenced above.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  to_netcdf() fails because of datetime encoding 374070147

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 2240.114ms · About: xarray-datasette