issue_comments

I thtew this pull request up before leaving the office so it would run all the tests. Turns out that np.can_cast is the wrong option here. It doesn't consider casting floats to integers safe even if all values are correctly representable. Another solution will have to be found.

I can investigate further on Monday, and combine this with the linked issues. Thanks for the extra context

-- Tim Heap @.***

On Thu, 11 May 2023, at 18:03, Kai Mühlbauer wrote:

@mx-moth https://github.com/mx-moth Yes, this casting should be fixed.

I'm adding a bit of context here, as this might need to be solved in combination with #7098 https://github.com/pydata/xarray/pull/7098 and #7827 https://github.com/pydata/xarray/pull/7827. #7098 https://github.com/pydata/xarray/pull/7098 removes undefined casting for decoding. In #7827 https://github.com/pydata/xarray/pull/7827 there are efforts to do this for encoding, too.

As cast_to_int_if_safe is called for encoding as well as decoding I'm not sure if all cases have been catched by these two PR.

One issue on decoding is that at least for datetime64 based times the calculated time_deltas are currently converted to float64 in the presence of NaT (although NaT can perfectly be expressed as int64). It would be great if you could try your PR on top of #7827 https://github.com/pydata/xarray/pull/7827 (which includes #7098 https://github.com/pydata/xarray/pull/7098) to see if that fixes the errors in this PR.

— Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/7834#issuecomment-1543526954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABAIM6WCCVWQRC7DY6YHKTXFSMMDANCNFSM6AAAAAAX5UZXZM. You are receiving this because you were mentioned.Message ID: @.***>

@mx-moth Yes, this casting should be fixed.

I'm adding a bit of context here, as this might need to be solved in combination with #7098 and #7827. #7098 removes undefined casting for decoding. In #7827 there are efforts to do this for encoding, too.

As cast_to_int_if_safe is called for encoding as well as decoding I'm not sure if all cases have been catched by these two PR.

One issue on decoding is that at least for datetime64 based times the calculated time_deltas are currently converted to float64 in the presence of NaT (although NaT can perfectly be expressed as int64). It would be great if you could try your PR on top of #7827 (which includes #7098) to see if that fixes the errors in this PR.

Using latest xarray and numpy >= 1.24.0, the following code generates a warning. This function is called when saving datasets to disk using dataset.to_netcdf:

python import numpy as np from xarray.coding.times import cast_to_int_if_safe array = np.array([1., 2., np.nan]) cast_to_int_if_safe(array)

$HOME/projects/xarray/xarray/coding/times.py:619: RuntimeWarning: invalid value encountered in cast int_num = np.asarray(num, dtype=np.int64) array([ 1., 2., nan])

Unsure if new tests need to be added, as the intention is that no behaviour changes except for the lack of warnings from numpy.