issues: 1419602897
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1419602897 | I_kwDOAMm_X85UnWvR | 7197 | Unstable pandas causes CF datetime64 issues | 1828519 | closed | 0 | 4 | 2022-10-23T02:30:39Z | 2022-10-26T16:00:35Z | 2022-10-26T16:00:35Z | CONTRIBUTOR | What happened?The Satpy project has a CI environment that installs numpy, pandas, and xarray (and a couple other packages) from their unstable sources (nightly builds, github source, etc). In the last week or two this environment has started failing with various datetime64 issues. It all seems to be caused by some recent change in pandas, but I can't place exactly what the problem is nor the commit/PR that started it. It seems there are a couple datetime related PRs. What did you expect to happen?Datetime or datetime64 objects should be allowed to be in whatever units they need to be in (days or minutes or nanoseconds. It seems parts of xarray (or pandas) assume Minimal Complete Verifiable ExampleYou should be able to take any environment with modern xarray and install dev pandas with:
Then run this snippet: ```Python import xarray as xr import numpy as np from xarray.coding.times import CFDatetimeCoder a = xr.DataArray(np.arange(1.0), dims=("time",), coords={"time": [np.datetime64('2018-05-30T10:05:00')]}) CFDatetimeCoder().encode(a.coords["time"]) ``` I haven't been able to generate a higher-level MVCE yet, but I'm hoping this little snippet will make the issue obvious to someone familiar with xarray internals. MVCE confirmation
Relevant log output``` ----> 1 CFDatetimeCoder().encode(a.coords["time"]) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:676, in CFDatetimeCoder.encode(self, variable, name) 672 dims, data, attrs, encoding = unpack_for_encoding(variable) 673 if np.issubdtype(data.dtype, np.datetime64) or contains_cftime_datetimes( 674 variable 675 ): --> 676 (data, units, calendar) = encode_cf_datetime( 677 data, encoding.pop("units", None), encoding.pop("calendar", None) 678 ) 679 safe_setitem(attrs, "units", units, name=name) 680 safe_setitem(attrs, "calendar", calendar, name=name) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:612, in encode_cf_datetime(dates, units, calendar) 609 dates = np.asarray(dates) 611 if units is None: --> 612 units = infer_datetime_units(dates) 613 else: 614 units = _cleanup_netcdf_time_units(units) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:394, in infer_datetime_units(dates) 392 print("Formatting datetime object") 393 reference_date = dates[0] if len(dates) > 0 else "1970-01-01" --> 394 reference_date = format_cftime_datetime(reference_date) 395 unique_timedeltas = np.unique(np.diff(dates)) 396 units = _infer_time_units_from_diff(unique_timedeltas) File ~/miniconda3/envs/satpy_py39_unstable/lib/python3.9/site-packages/xarray/coding/times.py:405, in format_cftime_datetime(date) 400 def format_cftime_datetime(date): 401 """Converts a cftime.datetime object to a string with the format: 402 YYYY-MM-DD HH:MM:SS.UUUUUU 403 """ 404 return "{:04d}-{:02d}-{:02d} {:02d}:{:02d}:{:02d}.{:06d}".format( --> 405 date.year, 406 date.month, 407 date.day, 408 date.hour, 409 date.minute, 410 date.second, 411 date.microsecond, 412 ) AttributeError: 'numpy.datetime64' object has no attribute 'year' ``` Anything else we need to know?No response Environment
```
INSTALLED VERSIONS
------------------
commit: None
python: 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:59)
[GCC 10.3.0]
python-bits: 64
OS: Linux
OS-release: 5.19.0-76051900-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: 4.8.1
xarray: 2022.10.0
pandas: 2.0.0.dev0+422.g6c46013c54
numpy: 1.23.4
scipy: 1.10.0.dev0+1848.f114d8b
netCDF4: 1.6.0
pydap: None
h5netcdf: 1.0.0
h5py: 3.7.0
Nio: None
zarr: 2.13.0a3.dev5
cftime: 1.6.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.4dev
cfgrib: None
iris: None
bottleneck: 1.3.5
dask: 2022.10.0+6.gc8dc3955
distributed: None
matplotlib: 3.7.0.dev473+gc450aa7baf
cartopy: 0.20.3
seaborn: None
numbagg: None
fsspec: 2022.5.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 65.3.0
pip: 22.2.2
conda: None
pytest: 7.1.1
IPython: 8.2.0
sphinx: 5.0.0
```
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/7197/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |