issue_comments

Very possibly. The first thing to look at is what opendap is sending:

http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc.das

@lesserwhirls @dennisHeimbigner Is there any reason to expect a difference between the downloaded file and the opendap view on TDS?

A temporary workaround without updating xarray might be to just edit the attributes to remove the offending unsigned attribute before decoding cf?

Sent from my iPhone

On Nov 30, 2018, at 1:35 PM, Deepak Cherian notifications@github.com wrote:

@jjspergel The single change in #2584 should fix your problem.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

@jjspergel The single change in #2584 should fix your problem.

The problem is that attrs['_Unsigned'] = 'false' which always triggers UnsignedIntegerCoder https://github.com/pydata/xarray/blob/0d6056e8816e3d367a64f36c7f1a5c4e1ce4ed4e/xarray/coding/variables.py#L269

That line expects unsigned to be bool and ends up casting the signed integer to an unsigned integer and things go haywire.

opendap is setting the _Unsigned attribute but it isn't present in the downloaded file.

ncdump -h http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc byte cloud_fraction(time, latitude, longitude) ; cloud_fraction:_Unsigned = "false" ; cloud_fraction:standard_name = "cloud_area_fraction" ; cloud_fraction:long_name = "cloud fraction computed over a 3x3 pixel array at the native resolution centered on this pixel" ; cloud_fraction:coordinates = "latitude longitude" ; cloud_fraction:actual_range = 0.f, 1.f ; cloud_fraction:valid_range = -127s, 127s ; cloud_fraction:scale_factor = 0.003937008f ; cloud_fraction:add_offset = 0.5f ; cloud_fraction:_FillValue = -128b ; cloud_fraction:scaled = 1b ; cloud_fraction:_ChunkSizes = 1, 361, 361 ;

ncdump -h patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc byte cloud_fraction_uncertainty(time, latitude, longitude) ; cloud_fraction_uncertainty:long_name = "cloud fraction uncertainty computed over a 3x3 array" ; cloud_fraction_uncertainty:coordinates = "latitude longitude" ; cloud_fraction_uncertainty:actual_range = 0.f, 1.f ; cloud_fraction_uncertainty:valid_range = -127b, 127b ; cloud_fraction_uncertainty:scale_factor = 0.003937008f ; cloud_fraction_uncertainty:add_offset = 0.5f ; cloud_fraction_uncertainty:_FillValue = -128b ; cloud_fraction_uncertainty:scaled = 1b ;

The CF decoding fails using opendap but not from the downloaded file. Pretty weird.

I can reproduce that on master. Thanks for the good example.

However, if I download the file, it seems to work OK. @jjspergel can you confirm that please?

EDIT:

Also, note that the scale_factor and add_offset attributes aren't set in the first ds.cloud_fraction repr but is in ds.cloud_fraction.encoding

``` {'source': 'http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc', 'original_shape': (1, 1800, 3600), 'dtype': dtype('int8'), '_Unsigned': 'false', '_FillValue': 128, 'scale_factor': 0.003937008,

'add_offset': 0.5, 'coordinates': 'latitude longitude'} ```

Yes @dcherian, when I downloaded it, and then imported into a notebook it seems to have decoded fine.

I can reproduce that on master. Thanks for the good example.

However, if I download the file, it seems to work OK. @jjspergel can you confirm that please?

EDIT:

Also, note that the scale_factor and add_offset attributes aren't set in the first ds.cloud_fraction repr but is in ds.cloud_fraction.encoding

{'source': 'http://www.ncei.noaa.gov/thredds/dodsC/avhrr-patmos-x-cloudprops-noaa-asc-fc/files/2003/patmosx_v05r03_NOAA-17_asc_d20030101_c20140314.nc', 'original_shape': (1, 1800, 3600), 'dtype': dtype('int8'), '_Unsigned': 'false', '_FillValue': 128, 'scale_factor': 0.003937008, 'add_offset': 0.5, 'coordinates': 'latitude longitude'}