home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 614275938

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
614275938 MDU6SXNzdWU2MTQyNzU5Mzg= 4045 Millisecond precision is lost on datetime64 during IO roundtrip 10676434 closed 0     9 2020-05-07T19:01:18Z 2021-01-03T23:39:04Z 2021-01-03T23:39:04Z NONE      

I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk.

MCVE Code Sample

bug_data.p.zip

Unzip the data. It will result in a pickle file.

```python bug_data_path = '/path/to/unzipped/bug_data.p' tmp_path = '~/Desktop/test.nc'

with open(bug_data_path, 'rb') as f: data = pickle.load(f)

selector = dict(animal=0, timepoint=0, wavelength='410', pair=0)

before_disk_ts = data.time.sel(**selector).values[()]

data.time.encoding = {'units': 'microseconds since 1900-01-01', 'calendar': 'proleptic_gregorian'}

data.to_netcdf(tmp_path) after_disk_ts = xr.load_dataarray(tmp_path).time.sel(**selector).values[()]

print(f'before roundtrip: {before_disk_ts}') print(f' after roundtrip: {after_disk_ts}') output: before roundtrip: 2017-02-22T16:24:10.586000000 after roundtrip: 2017-02-22T16:24:10.585999872 ```

Expected Output

Before: 2017-02-22T16:24:10.586000000 After: 2017-02-22T16:24:10.586000000

Problem Description

As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding).

Versions

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4045/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 9 rows from issue in issue_comments
Powered by Datasette · Queries took 0.863ms · About: xarray-datasette