issues: 386268842
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
386268842 | MDU6SXNzdWUzODYyNjg4NDI= | 2583 | decode_cf not scaling and off-setting correctly | 42973457 | closed | 0 | 8 | 2018-11-30T16:54:31Z | 2018-12-15T23:53:19Z | 2018-12-15T23:53:19Z | NONE | Problem descriptionI am importing atmospheric cloud fraction data from the PATMOSx OpenDap server (https://www.ncei.noaa.gov/thredds/satellite/avhrr-cloudprops-patmos-x.html). Cloud fraction is the percent of the pixel that the instruments flag as cloud, thus the output should be between 0 and 1. The data is stored unscaled as an int8. In order to retrieve the cloud fraction data, the data should be decoded as unscaled_data*scaling_factor + add_offset in order to be in the proper format. decode_cf does not seem to be using the algorithm in this way. ```python import xarray as xr url = 'http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc' Decoding using decode_cfds=xr.open_dataset(url, decode_cf=True) ds.cloud_fraction <xarray.DataArray 'cloud_fraction' (time: 1, latitude: 1800, longitude: 3600)> [6480000 values with dtype=float32] Coordinates: * latitude (latitude) float32 -89.947815 -89.84893 -89.74731 -89.64843 ... * longitude (longitude) float32 -179.94507 -179.84619 -179.7473 ... * time (time) datetime64[ns] 2003-01-01 Attributes: standard_name: cloud_area_fraction long_name: cloud fraction computed over a 3x3 pixel array at the nat... actual_range: [0. 1.] valid_range: [-127 127] scaled: 1 _ChunkSizes: [ 1 361 361] ds.cloud_fraction.plot() The cloud fraction data should be scaled to be a percentage between 0 and 1, but it is not.Manually decoding cloud_fractionds=xr.open_dataset(url, decode_cf=False) ds.cloud_fraction <xarray.DataArray 'cloud_fraction' (time: 1, latitude: 1800, longitude: 3600)> [6480000 values with dtype=int8] Coordinates: * latitude (latitude) int16 -32748 -32712 -32675 -32639 -32603 -32566 ... * longitude (longitude) int16 -32757 -32739 -32721 -32703 -32685 -32666 ... * time (time) float32 1041379200.0 Attributes: _Unsigned: false standard_name: cloud_area_fraction long_name: cloud fraction computed over a 3x3 pixel array at the nat... coordinates: latitude longitude actual_range: [0. 1.] valid_range: [-127 127] scale_factor: 0.003937008 add_offset: 0.5 _FillValue: -128 scaled: 1 _ChunkSizes: [ 1 361 361] cloud_fract_manual = ds.cloud_fraction * ds.cloud_fraction.scale_factor + ds.cloud_fraction.add_offset
cloud_fract_manual.plot()
```
decoded using decode_cf
Expected OutputDecoded manually Output of
|
{ "url": "https://api.github.com/repos/pydata/xarray/issues/2583/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | 13221727 | issue |