issues: 289853579

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
289853579 MDExOlB1bGxSZXF1ZXN0MTYzODc5NTc3 1840 Read small integers as float32, not float64 12229877 closed 0     4 2018-01-19T03:40:51Z 2018-04-19T02:50:25Z 2018-01-23T20:15:29Z CONTRIBUTOR   0 pydata/xarray/pulls/1840
  • [x] Closes #1842
  • [x] Tests added
  • [x] Tests passed
  • [x] Passes flake8 xarray (now part of tests)
  • [x] Fully documented, including whats-new.rst for all changes

Most satellites produce images with color depth in the range of eight to sixteen bits, which are therefore often stored as unsigned integers (with the quality mask in another variable). If you're lucky, they also have a scale_factor attribute and Xarray can automatically convert the integers to floats representing albedo.

This is fantastically convenient, and avoids all the bit-depth bugs from misremembered specifications. However, loading data as float64 when float32 is sufficient doubles memory usage in IO (even on multi-TB datasets...). While immediately downcasting helps, it's no substitute for doing the right thing first.

So this patch does some conservative checks, and if we can be sure float32 is safe we use that instead.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1840/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     13221727 pull

Links from other tables