issues: 343659822
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
343659822 | MDU6SXNzdWUzNDM2NTk4MjI= | 2304 | float32 instead of float64 when decoding int16 with scale_factor netcdf var using xarray | 971382 | closed | 0 | 32 | 2018-07-23T14:35:12Z | 2024-03-15T16:31:06Z | 2024-03-15T16:31:06Z | NONE | Code Sample :Considering a netcdf file file with the following variable:
Code: ```python from netCDF4 import Dataset import xarray as xr d = Dataset("test.nc") a = d.variables['agc_40hz'][:].flatten()[69] ## 21.940000000000001 'numpy.float64' x = xr.open_dataset("test.nc") b = x['agc_40hz'].values.flatten()[69] ## 21.939998626708984 'numpy.float32' abs(a - b) # 0.000001373291017 ``` Problem description :Different behaviour of xarray comparing to netCDF4 DatasetWhen reading the dataset with xarray we found that the decoded type was numpy.float32 instead of numpy.float64 This netcdf variable has an int16 dtype when the variable is read with the netCDF4 library directly, it is automatically converted to numpy.float64. in our case we loose on precision when using xarray. We found two solutions for this: First solution :This solution aims to prevent auto_maskandscale
Modification in xarray/backends/netCDF4_.py line 241 ```python def _disable_auto_decode_variable(var): """Disable automatic decoding on a netCDF4.Variable.
``` Second solution :This solution uses numpy.float64 whatever integer type provided.
Modification in xarray/core/dtypes.py line 85 ```python def maybe_promote(dtype): """Simpler equivalent of pandas.core.common._maybe_promote
``` Solution number 2 would be great for us. At this point we don't know if this modification would introduce some side effects. Is there another way to avoid this problem ? Expected OutputIn our case we expect the variable to be in numpy.float64 as it is done by netCDF4. Output of
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/2304/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |