html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/2304#issuecomment-410782982,https://api.github.com/repos/pydata/xarray/issues/2304,410782982,MDEyOklzc3VlQ29tbWVudDQxMDc4Mjk4Mg==,221526,2018-08-06T17:17:38Z,2018-08-06T17:17:38Z,CONTRIBUTOR,"Ah, ok, not scaling per-se (i.e. `* 0.01`), but a second round of value conversion. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,343659822 https://github.com/pydata/xarray/issues/2304#issuecomment-410779271,https://api.github.com/repos/pydata/xarray/issues/2304,410779271,MDEyOklzc3VlQ29tbWVudDQxMDc3OTI3MQ==,221526,2018-08-06T17:06:22Z,2018-08-06T17:06:22Z,CONTRIBUTOR,"I'm not following why the data are scaled twice. Your point about the rounding being different is well-taken, though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,343659822 https://github.com/pydata/xarray/issues/2304#issuecomment-410774955,https://api.github.com/repos/pydata/xarray/issues/2304,410774955,MDEyOklzc3VlQ29tbWVudDQxMDc3NDk1NQ==,221526,2018-08-06T16:52:42Z,2018-08-06T16:52:53Z,CONTRIBUTOR,"@shoyer But since it's a downstream calculation issue, and does not impact the actual precision of what's being read from the file, what's wrong with saying ""Use `data.astype(np.float64)`"". It's completely identical to doing it internally to xarray.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,343659822 https://github.com/pydata/xarray/issues/2304#issuecomment-410769706,https://api.github.com/repos/pydata/xarray/issues/2304,410769706,MDEyOklzc3VlQ29tbWVudDQxMDc2OTcwNg==,221526,2018-08-06T16:34:44Z,2018-08-06T16:36:16Z,CONTRIBUTOR,"A float32 values has 24 bits of precision in the significand, which is more than enough to store the 16-bits in in the original data; the exponent (8 bits) will more or less take care of the `* 0.01`: ```python >>> import numpy as np >>> np.float32(2194 * 0.01) 21.94 ``` What you're seeing is an artifact of printing out the values. I have no idea why something is printing out a float (only 7 decimal digits) out to 17 digits. Even float64 only has 16 digits (which is overkill for this application). The difference in subtracting the 32- and 64-bit values above are in the 8th decimal place, which is beyond the actual precision of the data; what you've just demonstrated is the difference in precision between 32-bit and 64-bit values, but it had nothing to do whatsoever with the data. If you're really worried about precision round-off for things like std. dev, you should probably calculate it using the raw integer values and scale afterwards. (I don't actually think this is necessary, though.)","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,343659822