html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1173#issuecomment-453804715,https://api.github.com/repos/pydata/xarray/issues/1173,453804715,MDEyOklzc3VlQ29tbWVudDQ1MzgwNDcxNQ==,2443309,2019-01-13T06:00:22Z,2019-01-13T06:00:22Z,MEMBER,Closing this old issue. We've taken care of most of the issues discussed here through the various backend updates over the past two years.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604
https://github.com/pydata/xarray/issues/1173#issuecomment-268705335,https://api.github.com/repos/pydata/xarray/issues/1173,268705335,MDEyOklzc3VlQ29tbWVudDI2ODcwNTMzNQ==,1217238,2016-12-22T02:51:29Z,2016-12-22T02:51:29Z,MEMBER,#1179 will make use of `SerializableLock()` for our default netCDF4/HDF5 lock.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604
https://github.com/pydata/xarray/issues/1173#issuecomment-268120151,https://api.github.com/repos/pydata/xarray/issues/1173,268120151,MDEyOklzc3VlQ29tbWVudDI2ODEyMDE1MQ==,306380,2016-12-20T00:36:32Z,2016-12-20T00:36:32Z,MEMBER,"There have been some efforts and progress in using many NetCDF files on a distributed POSIX filesystem (NFS, gluster, not HDFS) but there is still some pain here.  We should probably circle back up and figure out what still needs to be done (do you have a firm understanding of this @shoyer ?)

HDF5 on HDFS is, I suspect, sufficiently painful so that I would be more tempted to either avoid HDFS or to try other formats like ZArr (which I'm somewhat biased towards)  (cc @alimanfoo).  However my experience has been that most climate data lives on a POSIX file system, so experimentation here may not be high priority.

@JoyMonteiro if you have time then the first thing to do is to probably start using things and report where they're broken.  I'm confident that small things will present themselves quickly :)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604
https://github.com/pydata/xarray/issues/1173#issuecomment-268119475,https://api.github.com/repos/pydata/xarray/issues/1173,268119475,MDEyOklzc3VlQ29tbWVudDI2ODExOTQ3NQ==,306380,2016-12-20T00:32:40Z,2016-12-20T00:32:40Z,MEMBER,"Some related issues:

https://github.com/pydata/xarray/issues/798
https://github.com/dask/distributed/issues/629

Will write more shortly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604
https://github.com/pydata/xarray/issues/1173#issuecomment-268105839,https://api.github.com/repos/pydata/xarray/issues/1173,268105839,MDEyOklzc3VlQ29tbWVudDI2ODEwNTgzOQ==,1217238,2016-12-19T23:11:05Z,2016-12-19T23:11:05Z,MEMBER,I don't know if anyone has tested writing netCDFs with dask.distributed yet. I suspect the only immediate issue would be errors from dask because `threading.Lock()` can't be pickled. We need to switch to using dask's `SerializableLock()` to make dask.distributed work smoothly.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604
https://github.com/pydata/xarray/issues/1173#issuecomment-268103602,https://api.github.com/repos/pydata/xarray/issues/1173,268103602,MDEyOklzc3VlQ29tbWVudDI2ODEwMzYwMg==,1217238,2016-12-19T22:59:22Z,2016-12-19T22:59:22Z,MEMBER,"For (1), take a look at [save_mfdataset](http://xarray.pydata.org/en/stable/generated/xarray.save_mfdataset.html) for saving to multiple files.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,196541604