issue_comments

Table actions

2 rows where issue = 1419602897 and user = 1828519 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • djhoese · 2 ✖

issue 1

  • Unstable pandas causes CF datetime64 issues · 2 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1288100778 https://github.com/pydata/xarray/issues/7197#issuecomment-1288100778 https://api.github.com/repos/pydata/xarray/issues/7197 IC_kwDOAMm_X85Mxtuq djhoese 1828519 2022-10-23T12:23:07Z 2022-10-23T12:23:07Z CONTRIBUTOR

Ugh how did I miss that issue. Thanks. I'm fine with closing this since the existing tests have caught it.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Unstable pandas causes CF datetime64 issues 1419602897
1287986325 https://github.com/pydata/xarray/issues/7197#issuecomment-1287986325 https://api.github.com/repos/pydata/xarray/issues/7197 IC_kwDOAMm_X85MxRyV djhoese 1828519 2022-10-23T02:41:14Z 2022-10-23T02:41:14Z CONTRIBUTOR

Ah turns out doing to_netcdf on that DataArray is good enough:

a.to_netcdf("/tmp/mytest.nc")

to fail:

``` File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/core/dataarray.py:3752, in DataArray.to_netcdf(self, path, mode, format, group, engine, encoding, unlimited_dims, compute, invalid_netcdf) 3748 else: 3749 # No problems with the name - so we're fine! 3750 dataset = self.to_dataset() -> 3752 return to_netcdf( # type: ignore # mypy cannot resolve the overloads:( 3753 dataset, 3754 path, 3755 mode=mode, 3756 format=format, 3757 group=group, 3758 engine=engine, 3759 encoding=encoding, 3760 unlimited_dims=unlimited_dims, 3761 compute=compute, 3762 multifile=False, 3763 invalid_netcdf=invalid_netcdf, 3764 ) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/backends/api.py:1230, in to_netcdf(dataset, path_or_file, mode, format, group, engine, encoding, unlimited_dims, compute, multifile, invalid_netcdf) 1225 # TODO: figure out how to refactor this logic (here and in save_mfdataset) 1226 # to avoid this mess of conditionals 1227 try: 1228 # TODO: allow this work (setting up the file for writing array data) 1229 # to be parallelized with dask -> 1230 dump_to_store( 1231 dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims 1232 ) 1233 if autoclose: 1234 store.close() File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/backends/api.py:1277, in dump_to_store(dataset, store, writer, encoder, encoding, unlimited_dims) 1274 if encoder: 1275 variables, attrs = encoder(variables, attrs) -> 1277 store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/backends/common.py:266, in AbstractWritableDataStore.store(self, variables, attributes, check_encoding_set, writer, unlimited_dims) 263 if writer is None: 264 writer = ArrayWriter() --> 266 variables, attributes = self.encode(variables, attributes) 268 self.set_attributes(attributes) 269 self.set_dimensions(variables, unlimited_dims=unlimited_dims) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/backends/common.py:355, in WritableCFDataStore.encode(self, variables, attributes) 352 def encode(self, variables, attributes): 353 # All NetCDF files get CF encoded by default, without this attempting 354 # to write times, for example, would fail. --> 355 variables, attributes = cf_encoder(variables, attributes) 356 variables = {k: self.encode_variable(v) for k, v in variables.items()} 357 attributes = {k: self.encode_attribute(v) for k, v in attributes.items()} File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/conventions.py:868, in cf_encoder(variables, attributes) 865 # add encoding for time bounds variables if present. 866 _update_bounds_encoding(variables) --> 868 new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()} 870 # Remove attrs from bounds variables (issue #2921) 871 for var in new_vars.values(): File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/conventions.py:868, in <dictcomp>(.0) 865 # add encoding for time bounds variables if present. 866 _update_bounds_encoding(variables) --> 868 new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()} 870 # Remove attrs from bounds variables (issue #2921) 871 for var in new_vars.values(): File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/conventions.py:273, in encode_cf_variable(var, needs_copy, name) 264 ensure_not_multiindex(var, name=name) 266 for coder in [ 267 times.CFDatetimeCoder(), 268 times.CFTimedeltaCoder(), (...) 271 variables.UnsignedIntegerCoder(), 272 ]: --> 273 var = coder.encode(var, name=name) 275 # TODO(shoyer): convert all of these to use coders, too: 276 var = maybe_encode_nonstring_dtype(var, name=name) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:676, in CFDatetimeCoder.encode(self, variable, name) 672 dims, data, attrs, encoding = unpack_for_encoding(variable) 673 if np.issubdtype(data.dtype, np.datetime64) or contains_cftime_datetimes( 674 variable 675 ): --> 676 (data, units, calendar) = encode_cf_datetime( 677 data, encoding.pop("units", None), encoding.pop("calendar", None) 678 ) 679 safe_setitem(attrs, "units", units, name=name) 680 safe_setitem(attrs, "calendar", calendar, name=name) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:612, in encode_cf_datetime(dates, units, calendar) 609 dates = np.asarray(dates) 611 if units is None: --> 612 units = infer_datetime_units(dates) 613 else: 614 units = _cleanup_netcdf_time_units(units) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:394, in infer_datetime_units(dates) 392 print("Formatting datetime object") 393 reference_date = dates[0] if len(dates) > 0 else "1970-01-01" --> 394 reference_date = format_cftime_datetime(reference_date) 395 unique_timedeltas = np.unique(np.diff(dates)) 396 units = _infer_time_units_from_diff(unique_timedeltas) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:405, in format_cftime_datetime(date) 400 def format_cftime_datetime(date): 401 """Converts a cftime.datetime object to a string with the format: 402 YYYY-MM-DD HH:MM:SS.UUUUUU 403 """ 404 return "{:04d}-{:02d}-{:02d} {:02d}:{:02d}:{:02d}.{:06d}".format( --> 405 date.year, 406 date.month, 407 date.day, 408 date.hour, 409 date.minute, 410 date.second, 411 date.microsecond, 412 ) AttributeError: 'numpy.datetime64' object has no attribute 'year' ```

NOTE: Just because this clearly fails the check in this line:

https://github.com/pydata/xarray/blob/6cb97f645475bddf2f3b1e1a5f24f0f9de690683/xarray/coding/times.py#L384

It does not seem to be the only problem from my tests. Using datetime64 objects in various parts of a DataArray seem to cause different errors...I think. I need to do more tests.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   Unstable pandas causes CF datetime64 issues 1419602897

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);