html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/7827#issuecomment-1567109753,https://api.github.com/repos/pydata/xarray/issues/7827,1567109753,IC_kwDOAMm_X85daDJ5,6628425,2023-05-29T13:00:30Z,2023-05-29T13:00:30Z,MEMBER,"One other tricky edge case that occurs to me is one where an extreme fill value (e.g. `1e30`) is used for floating point fields. If we decode the times first, it might appear that the dates cannot be represented as nanosecond-precision values, but in reality they would be. We may need to think more about how to handle this edge case in addition to #7817.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1557089270,https://api.github.com/repos/pydata/xarray/issues/7827,1557089270,IC_kwDOAMm_X85cz0v2,6628425,2023-05-22T11:58:18Z,2023-05-22T11:58:18Z,MEMBER,"Great, yeah, that's a nice example without writing to disk. Indeed I saw those warnings too, but omitted them in my earlier message to focus on the encoding issue (sorry about that). I agree that these are something we should address.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1556891860,https://api.github.com/repos/pydata/xarray/issues/7827,1556891860,IC_kwDOAMm_X85czEjU,5821660,2023-05-22T09:40:04Z,2023-05-22T09:40:04Z,MEMBER,"The example below is only based on Variable and the cf encode/decode variable functions.
```python
import xarray as xr
import numpy as np
# create DataArray
times = [np.datetime64(""2000-01-01"", ""ns""), np.datetime64(""NaT"")]
da = xr.DataArray(times, dims=[""time""], name=""foo"")
da.encoding[""dtype""] = np.float64
da.encoding[""_FillValue""] = 20.0
# extract Variable
source_var = da.variable
print(""---------- source_var ------------------"")
print(source_var)
print(source_var.encoding)
# encode Variable
encoded_var = xr.conventions.encode_cf_variable(source_var)
print(""\n---------- encoded_var ------------------"")
print(encoded_var)
# decode Variable
decoded_var = xr.conventions.decode_cf_variable(""foo"", encoded_var)
print(""\n---------- decoded_var ------------------"")
print(decoded_var.load())
```
```python
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:618: RuntimeWarning: invalid value encountered in cast
int_num = np.asarray(num, dtype=np.int64)
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast
flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast
flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(
---------- source_var ------------------
array(['2000-01-01T00:00:00.000000000', 'NaT'],
dtype='datetime64[ns]')
{'dtype': , '_FillValue': 20.0}
dtype num float64
---------- encoded_var ------------------
array([ 0., 20.])
Attributes:
units: days since 2000-01-01 00:00:00
calendar: proleptic_gregorian
_FillValue: 20.0
---------- decoded_var ------------------
array(['2000-01-01T00:00:00.000000000', 'NaT'],
dtype='datetime64[ns]')
{'_FillValue': 20.0, 'units': 'days since 2000-01-01 00:00:00', 'calendar': 'proleptic_gregorian', 'dtype': dtype('float64')}
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1556869361,https://api.github.com/repos/pydata/xarray/issues/7827,1556869361,IC_kwDOAMm_X85cy_Dx,5821660,2023-05-22T09:24:47Z,2023-05-22T09:24:47Z,MEMBER,"@spencerkclark With current master I get the following `RuntimeWarning` running your code example:
- on encoding (calling `to_netcdf()`):
```python
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:618: RuntimeWarning: invalid value encountered in cast
int_num = np.asarray(num, dtype=np.int64)
```
- on decoding (calling `open_dataset()`):
```python
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast
flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(
/home/kai/miniconda/envs/xarray_311/lib/python3.11/site-packages/xarray/coding/times.py:254: RuntimeWarning: invalid value encountered in cast
flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(
```
The latter was discussed in #7098 (casting float64 to int64), the former was aimed to be resolved with this PR.
I'll try to create a test case using `Variable` and the respective encoding/decoding functions without involving IO (per your suggestion @spencerkclark).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1554532844,https://api.github.com/repos/pydata/xarray/issues/7827,1554532844,IC_kwDOAMm_X85cqEns,5821660,2023-05-19T12:57:31Z,2023-05-19T12:57:31Z,MEMBER,Thanks @spencerkclark for taking the time. NaN has been written to disk (as you assumed). Let's have another try next week.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1542767369,https://api.github.com/repos/pydata/xarray/issues/7827,1542767369,IC_kwDOAMm_X85b9MMJ,5821660,2023-05-10T20:27:08Z,2023-05-10T20:27:08Z,MEMBER,"@dcherian You were right from the beginning, changing order for decoding and handling `_FillValue` in `CFDatetimeCoder` seems to be one working solution with minimal code changes.
If the CI is happy I'll add tests to cover for the nanosecond issues in #7817. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1539356386,https://api.github.com/repos/pydata/xarray/issues/7827,1539356386,IC_kwDOAMm_X85bwLbi,5821660,2023-05-09T03:51:39Z,2023-05-09T03:51:39Z,MEMBER,"Thanks for the heads-up, @spencerkclark. No worries, I need to apply some changes anyway as it turns out.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1539086265,https://api.github.com/repos/pydata/xarray/issues/7827,1539086265,IC_kwDOAMm_X85bvJe5,6628425,2023-05-08T21:35:16Z,2023-05-08T21:35:16Z,MEMBER,Thanks @kmuehlbauer -- I just wanted to give you a heads up that I'm pretty busy this week. Hopefully I'll get a free moment to look at this more closely next week.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1538998850,https://api.github.com/repos/pydata/xarray/issues/7827,1538998850,IC_kwDOAMm_X85bu0JC,5821660,2023-05-08T20:22:28Z,2023-05-08T20:22:28Z,MEMBER,All tests have passed. Rebased now on latest main. The issue described in #7817 is resolved. Ready for first reviews. ,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1538966366,https://api.github.com/repos/pydata/xarray/issues/7827,1538966366,IC_kwDOAMm_X85busNe,5821660,2023-05-08T20:01:17Z,2023-05-08T20:01:17Z,MEMBER,"I've reset the order of coders to the initial behaviour. Instead the times are special cased in the CFMaskCoder. Locally it works, but I'll only trust the CI. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455
https://github.com/pydata/xarray/pull/7827#issuecomment-1538364933,https://api.github.com/repos/pydata/xarray/issues/7827,1538364933,IC_kwDOAMm_X85bsZYF,5821660,2023-05-08T13:29:07Z,2023-05-08T13:29:07Z,MEMBER,"@spencerkclark I'd appreciate if you could have a look here. All but one test pass, but I can't immediately see what that test is doing. Looks like mismatched dtypes on the attributes. If you have any suggestions how to possibly improve, please let me know. I've not added tests here, yet.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1700227455