home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 386268842

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
386268842 MDU6SXNzdWUzODYyNjg4NDI= 2583 decode_cf not scaling and off-setting correctly 42973457 closed 0     8 2018-11-30T16:54:31Z 2018-12-15T23:53:19Z 2018-12-15T23:53:19Z NONE      

Problem description

I am importing atmospheric cloud fraction data from the PATMOSx OpenDap server (https://www.ncei.noaa.gov/thredds/satellite/avhrr-cloudprops-patmos-x.html). Cloud fraction is the percent of the pixel that the instruments flag as cloud, thus the output should be between 0 and 1. The data is stored unscaled as an int8. In order to retrieve the cloud fraction data, the data should be decoded as unscaled_data*scaling_factor + add_offset in order to be in the proper format. decode_cf does not seem to be using the algorithm in this way.

```python import xarray as xr url = 'http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc'

Decoding using decode_cf

ds=xr.open_dataset(url, decode_cf=True) ds.cloud_fraction <xarray.DataArray 'cloud_fraction' (time: 1, latitude: 1800, longitude: 3600)> [6480000 values with dtype=float32] Coordinates: * latitude (latitude) float32 -89.947815 -89.84893 -89.74731 -89.64843 ... * longitude (longitude) float32 -179.94507 -179.84619 -179.7473 ... * time (time) datetime64[ns] 2003-01-01 Attributes: standard_name: cloud_area_fraction long_name: cloud fraction computed over a 3x3 pixel array at the nat... actual_range: [0. 1.] valid_range: [-127 127] scaled: 1 _ChunkSizes: [ 1 361 361]

ds.cloud_fraction.plot()

The cloud fraction data should be scaled to be a percentage between 0 and 1, but it is not.

Manually decoding cloud_fraction

ds=xr.open_dataset(url, decode_cf=False)

ds.cloud_fraction <xarray.DataArray 'cloud_fraction' (time: 1, latitude: 1800, longitude: 3600)> [6480000 values with dtype=int8] Coordinates: * latitude (latitude) int16 -32748 -32712 -32675 -32639 -32603 -32566 ... * longitude (longitude) int16 -32757 -32739 -32721 -32703 -32685 -32666 ... * time (time) float32 1041379200.0 Attributes: _Unsigned: false standard_name: cloud_area_fraction long_name: cloud fraction computed over a 3x3 pixel array at the nat... coordinates: latitude longitude actual_range: [0. 1.] valid_range: [-127 127] scale_factor: 0.003937008 add_offset: 0.5 _FillValue: -128 scaled: 1 _ChunkSizes: [ 1 361 361]

cloud_fract_manual = ds.cloud_fraction * ds.cloud_fraction.scale_factor + ds.cloud_fraction.add_offset cloud_fract_manual.plot() ``` decoded using decode_cf

Expected Output

Decoded manually ds.cloud_fraction * ds.cloud_fraction.scale_factor + ds.cloud_fraction.add_offset

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.6.final.0 python-bits: 64 OS: Linux OS-release: 4.4.111+ machine: x86_64 processor: x86_64 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.8 pandas: 0.23.2 numpy: 1.15.1 scipy: 1.1.0 netCDF4: 1.4.2 h5netcdf: 0.6.2 h5py: 2.8.0 Nio: None zarr: 2.2.0 bottleneck: 1.2.1 cyordereddict: None dask: 0.18.2 distributed: 1.22.1 matplotlib: 2.2.3 cartopy: 0.17.0 seaborn: None setuptools: 39.2.0 pip: 18.1 conda: 4.5.4 pytest: 3.9.1 IPython: 6.4.0 sphinx: None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2583/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 8 rows from issue in issue_comments
Powered by Datasette · Queries took 0.633ms · About: xarray-datasette