html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/2503#issuecomment-481222036,https://api.github.com/repos/pydata/xarray/issues/2503,481222036,MDEyOklzc3VlQ29tbWVudDQ4MTIyMjAzNg==,1197350,2019-04-09T12:02:01Z,2019-04-09T12:02:01Z,MEMBER,This works with latest libraries.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-453866603,https://api.github.com/repos/pydata/xarray/issues/2503,453866603,MDEyOklzc3VlQ29tbWVudDQ1Mzg2NjYwMw==,2443309,2019-01-13T21:20:51Z,2019-01-13T21:20:51Z,MEMBER,"@rabernat - do think this was resolved? If I'm understanding the thread correctly, it seems this was a libnetcdf version issue. Feel free to reopen if I've got that wrong.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432796835,https://api.github.com/repos/pydata/xarray/issues/2503,432796835,MDEyOklzc3VlQ29tbWVudDQzMjc5NjgzNQ==,950575,2018-10-24T19:29:11Z,2018-10-24T19:29:11Z,CONTRIBUTOR,"> h10edf3e_1 contains the timeout fix and is build against hdf5 1.10.2. The `conda-forge` h9cd6fdc_11 build is against hdf5 1.10.3 perhaps that makes a different?
There are many variables at play here. The env that solved it in https://github.com/pydata/xarray/issues/2503#issuecomment-432645477 seems quite different from the env where the problem happened, including an `xarray` dev version. I'm not sure `hdf5` is a good candidate to blame :smile:","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432788686,https://api.github.com/repos/pydata/xarray/issues/2503,432788686,MDEyOklzc3VlQ29tbWVudDQzMjc4ODY4Ng==,1050278,2018-10-24T19:04:44Z,2018-10-24T19:04:44Z,CONTRIBUTOR,h10edf3e_1 contains the timeout fix and is build against hdf5 1.10.2. The `conda-forge` h9cd6fdc_11 build is against hdf5 1.10.3 perhaps that makes a different?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432784219,https://api.github.com/repos/pydata/xarray/issues/2503,432784219,MDEyOklzc3VlQ29tbWVudDQzMjc4NDIxOQ==,950575,2018-10-24T18:53:22Z,2018-10-24T18:53:22Z,CONTRIBUTOR,"> In `defaults` libnetcdf4 4.6.1 build 1 and above contain the timeout fix, build 0 has the original timeout.
Thanks @jjhelmus! I guess that info and https://github.com/pydata/xarray/issues/2503#issuecomment-432483817 eliminates the timeout issue from the equation.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432783232,https://api.github.com/repos/pydata/xarray/issues/2503,432783232,MDEyOklzc3VlQ29tbWVudDQzMjc4MzIzMg==,1050278,2018-10-24T18:50:48Z,2018-10-24T18:50:48Z,CONTRIBUTOR,"In `defaults` libnetcdf4 4.6.1 build 1 and above contain the timeout fix, build 0 has the original timeout.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432747410,https://api.github.com/repos/pydata/xarray/issues/2503,432747410,MDEyOklzc3VlQ29tbWVudDQzMjc0NzQxMA==,221526,2018-10-24T17:13:14Z,2018-10-24T17:13:14Z,CONTRIBUTOR,"Oh, I didn't even catch that the original was on defaults.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432746421,https://api.github.com/repos/pydata/xarray/issues/2503,432746421,MDEyOklzc3VlQ29tbWVudDQzMjc0NjQyMQ==,950575,2018-10-24T17:10:44Z,2018-10-24T17:10:44Z,CONTRIBUTOR,"> That version has the fix for the issue.
I know that @jjhelmus ported the fix to `defaults` but I'm not sure which build number has it, and/or if the previous one was remove, b/c defaults builds are not as transparent as `conda-forge`'s :smile:
He can probably say more about that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432744441,https://api.github.com/repos/pydata/xarray/issues/2503,432744441,MDEyOklzc3VlQ29tbWVudDQzMjc0NDQ0MQ==,221526,2018-10-24T17:06:01Z,2018-10-24T17:06:01Z,CONTRIBUTOR,That version has the fix for the issue.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432743208,https://api.github.com/repos/pydata/xarray/issues/2503,432743208,MDEyOklzc3VlQ29tbWVudDQzMjc0MzIwOA==,1872600,2018-10-24T17:02:34Z,2018-10-24T17:02:34Z,NONE,"The version that is working in [@rabernat's esgf binder env](https://github.com/rabernat/pangeo_esgf_demo/blob/master/binder/environment.yml) is:
```
libnetcdf 4.6.1 h9cd6fdc_11 conda-forge
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432739449,https://api.github.com/repos/pydata/xarray/issues/2503,432739449,MDEyOklzc3VlQ29tbWVudDQzMjczOTQ0OQ==,221526,2018-10-24T16:54:05Z,2018-10-24T16:54:05Z,CONTRIBUTOR,"The original version of libnetcdf in @rabernat 's environment definitely had the opendap timeout issue. Not sure if that's the root cause of the problem, or not, but it's suspect.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432706068,https://api.github.com/repos/pydata/xarray/issues/2503,432706068,MDEyOklzc3VlQ29tbWVudDQzMjcwNjA2OA==,1872600,2018-10-24T15:27:33Z,2018-10-24T15:27:33Z,NONE,"I fired up my notebook on @rabernat's binder env and it worked fine also:
https://nbviewer.jupyter.org/gist/rsignell-usgs/aebdac44a1d773b99673cb132c2ef5eb","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432645477,https://api.github.com/repos/pydata/xarray/issues/2503,432645477,MDEyOklzc3VlQ29tbWVudDQzMjY0NTQ3Nw==,1197350,2018-10-24T13:01:00Z,2018-10-24T13:01:00Z,MEMBER,"I created a binderized version of this issue with the latest dev xarray and fresh installs of all other packages:
https://github.com/rabernat/pangeo_esgf_demo
It appears to work fine!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432483817,https://api.github.com/repos/pydata/xarray/issues/2503,432483817,MDEyOklzc3VlQ29tbWVudDQzMjQ4MzgxNw==,1197350,2018-10-24T01:56:10Z,2018-10-24T01:56:10Z,MEMBER,"```
$ conda list libnetcdf
# packages in environment at /opt/conda:
#
# Name Version Build Channel
libnetcdf 4.6.1 h10edf3e_1 defaults
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432422763,https://api.github.com/repos/pydata/xarray/issues/2503,432422763,MDEyOklzc3VlQ29tbWVudDQzMjQyMjc2Mw==,221526,2018-10-23T21:16:05Z,2018-10-23T21:16:16Z,CONTRIBUTOR,"@lesserwhirls That's an interesting idea. (@rsignell-usgs That's the one.)
@rabernat What version of the conda-forge libnetcdf package is deployed wherever you're running?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432416210,https://api.github.com/repos/pydata/xarray/issues/2503,432416210,MDEyOklzc3VlQ29tbWVudDQzMjQxNjIxMA==,1217238,2018-10-23T20:55:57Z,2018-10-23T20:55:57Z,MEMBER,@rabernat have you tried using the development version of xarray? I think we fixed a few serialization/ netCDF4 bugs with the backends refactor.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432416114,https://api.github.com/repos/pydata/xarray/issues/2503,432416114,MDEyOklzc3VlQ29tbWVudDQzMjQxNjExNA==,1872600,2018-10-23T20:55:42Z,2018-10-23T20:55:42Z,NONE,"@lesserwhirls , is this the issue you are referring to? https://github.com/Unidata/netcdf4-python/issues/836","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432415704,https://api.github.com/repos/pydata/xarray/issues/2503,432415704,MDEyOklzc3VlQ29tbWVudDQzMjQxNTcwNA==,1872600,2018-10-23T20:54:24Z,2018-10-23T20:54:24Z,NONE,"@jhamman, doesn't this dask status plot tell us that multiple workers are connecting and getting data?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432396648,https://api.github.com/repos/pydata/xarray/issues/2503,432396648,MDEyOklzc3VlQ29tbWVudDQzMjM5NjY0OA==,67096,2018-10-23T19:59:30Z,2018-10-23T19:59:30Z,NONE,"> So for just a short time period (in this case 50 time steps, 2 chunks in time), it would usually work:
The ""short time period"" makes me wonder...@dopplershift - could this be due to the netCDF-C / curl timeout issue you mentioned today?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432390876,https://api.github.com/repos/pydata/xarray/issues/2503,432390876,MDEyOklzc3VlQ29tbWVudDQzMjM5MDg3Ng==,2443309,2018-10-23T19:41:41Z,2018-10-23T19:41:41Z,MEMBER,@rsignell-usgs - are you able to tell if multiple processes (workers) have authenticated on the server side? I think this detail would really help us isolate the problem.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432389980,https://api.github.com/repos/pydata/xarray/issues/2503,432389980,MDEyOklzc3VlQ29tbWVudDQzMjM4OTk4MA==,1872600,2018-10-23T19:39:09Z,2018-10-23T19:39:09Z,NONE,"Perhaps it's also worth mentioning that I don't see any errors on the THREDDS server side on either the tomcat catalina or thredds threddsServlet logs. @lesserwhirls, any ideas?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432374559,https://api.github.com/repos/pydata/xarray/issues/2503,432374559,MDEyOklzc3VlQ29tbWVudDQzMjM3NDU1OQ==,1872600,2018-10-23T18:53:28Z,2018-10-23T19:39:08Z,NONE,"FWIW, in my workflow there was nothing fundamentally wrong, meaning that the requests worked for a while, but eventually would die with the `NetCDF: Malformed or inaccessible DAP DDS` message.
So for just a short time period (in this case 50 time steps, 2 chunks in time), it would usually work:
https://nbviewer.jupyter.org/gist/rsignell-usgs/1155c76ed3440858ced8132e4cd81df4
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432367931,https://api.github.com/repos/pydata/xarray/issues/2503,432367931,MDEyOklzc3VlQ29tbWVudDQzMjM2NzkzMQ==,1872600,2018-10-23T18:34:48Z,2018-10-23T19:18:52Z,NONE,"I tried a similar workflow last week with an AWS kubernetes cluster with opendap endpoints and it also failed: https://nbviewer.jupyter.org/gist/rsignell-usgs/8583ea8f8b5e1c926b0409bd536095a9
I thought it was likely some intermittent problem that wasn't handled well. In my case after a while I get:
```
distributed.worker - WARNING - Compute Failed Function: getter args: (ImplicitToExplicitIndexingAdapter(array=CopyOnWriteArray(array=LazilyOuterIndexedArray(array=_ElementwiseFunctionArray(LazilyOuterIndexedArray(array=, key=BasicIndexer((slice(None, None, None), slice(None, None, None), slice(None, None, None), slice(None, None, None)))), func=functools.partial(, encoded_fill_values={1e+37}, decoded_fill_value=nan, dtype=dtype('float64')), dtype=dtype('float64')), key=BasicIndexer((slice(None, None, None), slice(None, None, None), slice(None, None, None), slice(None, None, None)))))), (slice(375, 400, None), slice(0, 7, None), slice(0, 670, None), slice(0, 300, None))) kwargs: {} Exception: OSError(-72, 'NetCDF: Malformed or inaccessible DAP DDS')
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432370887,https://api.github.com/repos/pydata/xarray/issues/2503,432370887,MDEyOklzc3VlQ29tbWVudDQzMjM3MDg4Nw==,221526,2018-10-23T18:43:23Z,2018-10-23T18:43:23Z,CONTRIBUTOR,"Just so I'm clear on how the workflow looks:
1. Open dataset with NetCDF/OPeNDAP
2. Serialize NetCDFDataStore (pickle? netcdf file?)
3. Ship to Dask workers
4. Reconstitute NetCDFDataStore
Certainly does seem like there's something stale in what the remote workers are getting. Confused why it works for the others, though.
I can prioritize this a bit and dig in to see what I can figure out--though I'm teaching through tomorrow. May be able to dig into this while at ECMWF.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432355234,https://api.github.com/repos/pydata/xarray/issues/2503,432355234,MDEyOklzc3VlQ29tbWVudDQzMjM1NTIzNA==,1197350,2018-10-23T18:01:39Z,2018-10-23T18:01:39Z,MEMBER,"This is fairly high priority for me, as it relates to the ongoing project to access CMIP6 data from an ESGF node running in google cloud (https://github.com/pangeo-data/pangeo/issues/420).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666
https://github.com/pydata/xarray/issues/2503#issuecomment-432354151,https://api.github.com/repos/pydata/xarray/issues/2503,432354151,MDEyOklzc3VlQ29tbWVudDQzMjM1NDE1MQ==,2443309,2018-10-23T17:58:46Z,2018-10-23T17:58:46Z,MEMBER,"I'm wondering if there is some authentication that is not being properly distributed to the workers. I'm not actually sure how opendap works in this case. Perhaps @dopplershift has some ideas here? Has anyone, maybe @rsignell-usgs, used the kubernetes cluster with opendap endpoints?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,373121666