home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

8 rows where author_association = "MEMBER", issue = 1700227455 and user = 5821660 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • kmuehlbauer · 8 ✖

issue 1

  • Preserve nanosecond resolution when encoding/decoding times · 8 ✖

author_association 1

  • MEMBER · 8 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1556891860 https://github.com/pydata/xarray/pull/7827#issuecomment-1556891860 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85czEjU kmuehlbauer 5821660 2023-05-22T09:40:04Z 2023-05-22T09:40:04Z MEMBER

The example below is only based on Variable and the cf encode/decode variable functions.

```python import xarray as xr import numpy as np

create DataArray

times = [np.datetime64("2000-01-01", "ns"), np.datetime64("NaT")] da = xr.DataArray(times, dims=["time"], name="foo") da.encoding["dtype"] = np.float64 da.encoding["_FillValue"] = 20.0

extract Variable

source_var = da.variable print("---------- source_var ------------------") print(source_var) print(source_var.encoding)

encode Variable

encoded_var = xr.conventions.encode_cf_variable(source_var) print("\n---------- encoded_var ------------------") print(encoded_var)

decode Variable

decoded_var = xr.conventions.decode_cf_variable("foo", encoded_var) print("\n---------- decoded_var ------------------") print(decoded_var.load()) ```

```python /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:618: RuntimeWarning: invalid value encountered in cast int_num = np.asarray(num, dtype=np.int64) /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype( /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(

---------- source_var ------------------ <xarray.Variable (time: 2)> array(['2000-01-01T00:00:00.000000000', 'NaT'], dtype='datetime64[ns]') {'dtype': <class 'numpy.float64'>, '_FillValue': 20.0} dtype num float64

---------- encoded_var ------------------ <xarray.Variable (time: 2)> array([ 0., 20.]) Attributes: units: days since 2000-01-01 00:00:00 calendar: proleptic_gregorian _FillValue: 20.0

---------- decoded_var ------------------ <xarray.Variable (time: 2)> array(['2000-01-01T00:00:00.000000000', 'NaT'], dtype='datetime64[ns]') {'_FillValue': 20.0, 'units': 'days since 2000-01-01 00:00:00', 'calendar': 'proleptic_gregorian', 'dtype': dtype('float64')} ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1556869361 https://github.com/pydata/xarray/pull/7827#issuecomment-1556869361 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85cy_Dx kmuehlbauer 5821660 2023-05-22T09:24:47Z 2023-05-22T09:24:47Z MEMBER

@spencerkclark With current master I get the following RuntimeWarning running your code example:

  • on encoding (calling to_netcdf()):

python /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:618: RuntimeWarning: invalid value encountered in cast int_num = np.asarray(num, dtype=np.int64)

  • on decoding (calling open_dataset()):

python /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype( /home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(

The latter was discussed in #7098 (casting float64 to int64), the former was aimed to be resolved with this PR.

I'll try to create a test case using Variable and the respective encoding/decoding functions without involving IO (per your suggestion @spencerkclark).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1554532844 https://github.com/pydata/xarray/pull/7827#issuecomment-1554532844 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85cqEns kmuehlbauer 5821660 2023-05-19T12:57:31Z 2023-05-19T12:57:31Z MEMBER

Thanks @spencerkclark for taking the time. NaN has been written to disk (as you assumed). Let's have another try next week.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1542767369 https://github.com/pydata/xarray/pull/7827#issuecomment-1542767369 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85b9MMJ kmuehlbauer 5821660 2023-05-10T20:27:08Z 2023-05-10T20:27:08Z MEMBER

@dcherian You were right from the beginning, changing order for decoding and handling _FillValue in CFDatetimeCoder seems to be one working solution with minimal code changes.

If the CI is happy I'll add tests to cover for the nanosecond issues in #7817.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1539356386 https://github.com/pydata/xarray/pull/7827#issuecomment-1539356386 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85bwLbi kmuehlbauer 5821660 2023-05-09T03:51:39Z 2023-05-09T03:51:39Z MEMBER

Thanks for the heads-up, @spencerkclark. No worries, I need to apply some changes anyway as it turns out.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1538998850 https://github.com/pydata/xarray/pull/7827#issuecomment-1538998850 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85bu0JC kmuehlbauer 5821660 2023-05-08T20:22:28Z 2023-05-08T20:22:28Z MEMBER

All tests have passed. Rebased now on latest main. The issue described in #7817 is resolved. Ready for first reviews.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1538966366 https://github.com/pydata/xarray/pull/7827#issuecomment-1538966366 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85busNe kmuehlbauer 5821660 2023-05-08T20:01:17Z 2023-05-08T20:01:17Z MEMBER

I've reset the order of coders to the initial behaviour. Instead the times are special cased in the CFMaskCoder. Locally it works, but I'll only trust the CI.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455
1538364933 https://github.com/pydata/xarray/pull/7827#issuecomment-1538364933 https://api.github.com/repos/pydata/xarray/issues/7827 IC_kwDOAMm_X85bsZYF kmuehlbauer 5821660 2023-05-08T13:29:07Z 2023-05-08T13:29:07Z MEMBER

@spencerkclark I'd appreciate if you could have a look here. All but one test pass, but I can't immediately see what that test is doing. Looks like mismatched dtypes on the attributes. If you have any suggestions how to possibly improve, please let me know. I've not added tests here, yet.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Preserve nanosecond resolution when encoding/decoding times 1700227455

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 985.557ms · About: xarray-datasette