html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/822#issuecomment-464950816,https://api.github.com/repos/pydata/xarray/issues/822,464950816,MDEyOklzc3VlQ29tbWVudDQ2NDk1MDgxNg==,2448579,2019-02-19T02:11:31Z,2019-02-19T02:11:31Z,MEMBER,Fixed upstream.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-208706109,https://api.github.com/repos/pydata/xarray/issues/822,208706109,MDEyOklzc3VlQ29tbWVudDIwODcwNjEwOQ==,1217238,2016-04-12T04:57:47Z,2016-04-12T04:57:47Z,MEMBER,"@forman Just a note -- if h5py can read the data correctly, you can read the data into xarray using [h5netcdf](https://github.com/shoyer/h5netcdf) (with `engine='h5netcdf'` in open_dataset). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-208237051,https://api.github.com/repos/pydata/xarray/issues/822,208237051,MDEyOklzc3VlQ29tbWVudDIwODIzNzA1MQ==,10050469,2016-04-11T08:53:43Z,2016-04-11T08:54:20Z,MEMBER,"Note that `ncview` also displays the output correctly. It seems that the problem occurs at the necdf4 level already: ``` python import netCDF4 import matplotlib.pyplot as plt f = '/home/mowglie/Downloads/20100101120000-ESACCI-L4_GHRSST-SSTdepth-OSTIA-GLOB_LT-v02.0-fv01.1.nc' d = netCDF4.Dataset(f) v = d['analysed_sst'] v.set_auto_maskandscale(False) plt.imshow(v[0, ...], origin='lower') plt.show() ``` ![selection_071](https://cloud.githubusercontent.com/assets/10050469/14421752/94bee3e0-ffd3-11e5-97a7-224469df1976.png) To be compared to the equivalent output from [IDL](https://en.wikipedia.org/wiki/IDL_%28programming_language%29) at the netcdf backend level: ![selection_070](https://cloud.githubusercontent.com/assets/10050469/14421721/790ba00c-ffd3-11e5-9ffd-32daa03849df.png) ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-208088579,https://api.github.com/repos/pydata/xarray/issues/822,208088579,MDEyOklzc3VlQ29tbWVudDIwODA4ODU3OQ==,1217238,2016-04-10T23:14:50Z,2016-04-10T23:14:50Z,MEMBER,"I just opened this up with `decode_cf=False`. Here's what the raw data looks like: ``` ds.analysed_sst[0].plot.imshow() ``` ![image](https://cloud.githubusercontent.com/assets/1217238/14413677/2a1a0b7a-ff36-11e5-9de2-e7ee1c236c18.png) with attributes: ``` Attributes: _FillValue: -32768 units: kelvin scale_factor: 0.01 add_offset: 273.15 long_name: analysed sea surface temperature valid_min: -300 valid_max: 4500 standard_name: sea_water_temperature depth: 20 cm source: ATSR<1,2>-ESACCI-L3U-v1.0, AATSR-ESACCI-L3U-v1.0, AVHRR<12,14,15,16,17,18>_G-ESACCI-L2P-v1.0, AVHRRMTA-ESACCI-L2P-v1.0 comment: SST analysis produced for ESA SST CCI project using the OSTIA system in reanalysis mode. ``` To me, It looks like somebody just mis-prepared this dataset, given the smooth transitions in the ocean from dark red to dark blue, which would correspond to numeric overflow. If not, there are lots of places in the ocean where the temperature is in negative degrees Kelvin. I don't have a strong opinion on whether or not we should automatically masking values outside `valid_min`/`valid_max`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-207552831,https://api.github.com/repos/pydata/xarray/issues/822,207552831,MDEyOklzc3VlQ29tbWVudDIwNzU1MjgzMQ==,1217238,2016-04-08T18:46:49Z,2016-04-08T18:46:49Z,MEMBER,"So we actually use our own value scaling logic, independent of netCDF4. It's interesting that we have the same issue, though! Possibly it's because we don't use the `valid_min`/`valid_max` arguments in determining what should be masked? I can download your dataset later and take a more in depth look. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-207510870,https://api.github.com/repos/pydata/xarray/issues/822,207510870,MDEyOklzc3VlQ29tbWVudDIwNzUxMDg3MA==,10050469,2016-04-08T16:51:25Z,2016-04-08T16:51:25Z,MEMBER,"For what its worth, I've tested your file with my IDL library, and I get: ``` IDL> print, min(data), max(data) -54.5300 306.440 IDL> print, TIME_to_STR(time) 01.01.2010 12:00:00 ``` you should fill a report in netcdf4 as @rabernat suggests. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-207510306,https://api.github.com/repos/pydata/xarray/issues/822,207510306,MDEyOklzc3VlQ29tbWVudDIwNzUxMDMwNg==,1197350,2016-04-08T16:50:22Z,2016-04-08T16:50:22Z,MEMBER,"@fmaussion then it's a netCDF bug https://github.com/Unidata/netcdf4-python/issues ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644 https://github.com/pydata/xarray/issues/822#issuecomment-207508190,https://api.github.com/repos/pydata/xarray/issues/822,207508190,MDEyOklzc3VlQ29tbWVudDIwNzUwODE5MA==,10050469,2016-04-08T16:43:47Z,2016-04-08T16:43:47Z,MEMBER,"It seems that the problem is happening at the NetCDF4 level already: ``` python import netCDF4 import numpy as np f = '/home/mowglie/Downloads/20100101120000-ESACCI-L4_GHRSST-SSTdepth-OSTIA-GLOB_LT-v02.0-fv01.1.nc' d = netCDF4.Dataset(f) da = d['analysed_sst'][:] print(np.max(da), np.min(da)) print(netCDF4.num2date(d['time'][:], d['time'].units)) ``` prints ``` 600.82 -54.53 [datetime.datetime(1947, 5, 12, 9, 58, 14)] ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,146975644