html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/2494#issuecomment-433393215,https://api.github.com/repos/pydata/xarray/issues/2494,433393215,MDEyOklzc3VlQ29tbWVudDQzMzM5MzIxNQ==,32069530,2018-10-26T12:37:30Z,2018-10-26T12:37:30Z,NONE,"Hi all, I finally figured out my problem. On each independent process xr.open_mfdataset() seems to naturally try to do some multi-threaded access (even without parallel option ?). Each node of my cluster was configured in such a way that multi-threading was possible (my mistake). Here was my yaml config file used by PBSCluster() ``` jobqueue: pbs: name: dask-worker # Dask worker options cores: 56 processes: 28 ``` I tough that the parallel=True option was to enable parallelized access for my independent process. It actually enable parallelized access for possible threads of each process. Now, I have removed parallel=True from xr.open_mfdataset() call and ensure 1 thread by process by changing my config file: ``` cores: 28 processes: 28 ``` Thanks again for your help ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,371906566 https://github.com/pydata/xarray/issues/2494#issuecomment-431796693,https://api.github.com/repos/pydata/xarray/issues/2494,431796693,MDEyOklzc3VlQ29tbWVudDQzMTc5NjY5Mw==,32069530,2018-10-22T10:27:04Z,2018-10-22T10:27:04Z,NONE,"@jhamman I was aware of the difference between the two parallel options. I was thus wondering if I could pass a parallel option to the netcdf4 library via the open_mfdataset() call. I tried to change the engine to netcdf4 and added the backend_kwarg : `backend_kwargs={'parallel':True}` but I get the same error. I 'll try the suggestion of Stephan to see how it behaves and I will report back. Thanks","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,371906566