html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/672#issuecomment-162463237,https://api.github.com/repos/pydata/xarray/issues/672,162463237,MDEyOklzc3VlQ29tbWVudDE2MjQ2MzIzNw==,7300413,2015-12-07T09:33:17Z,2015-12-07T09:33:17Z,NONE,"You were right, my chunk sizes were too large. It did not matter how many threads dask used either (4 vs. 8). The I/O component is still high, but that is also because I'm writing the final computed DataArray to disk. Thanks! ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,120681918 https://github.com/pydata/xarray/issues/672#issuecomment-162426520,https://api.github.com/repos/pydata/xarray/issues/672,162426520,MDEyOklzc3VlQ29tbWVudDE2MjQyNjUyMA==,1217238,2015-12-07T06:46:40Z,2015-12-07T06:46:40Z,MEMBER,"Those sorts of operations should be easily parallelized, although depending on what you're doing with the data they might also be IO bound. It's worth experimenting with chunk sizes. For control on the number of threads, see his page: http://dask.pydata.org/en/latest/scheduler-overview.html#configuring-the-schedulers ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,120681918 https://github.com/pydata/xarray/issues/672#issuecomment-162419595,https://api.github.com/repos/pydata/xarray/issues/672,162419595,MDEyOklzc3VlQ29tbWVudDE2MjQxOTU5NQ==,7300413,2015-12-07T06:11:39Z,2015-12-07T06:11:39Z,NONE,"Hello, I ran it with the dask profiler, and I looked at the top output disaggregated by core. It does seem to use multiple cores, but it seems to be using 8 threads when I looked at prof.visualize() (hyperthreading :P) and I feel this is killing performance. How can I control how many threads to use? Thanks, Joy ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,120681918 https://github.com/pydata/xarray/issues/672#issuecomment-162417283,https://api.github.com/repos/pydata/xarray/issues/672,162417283,MDEyOklzc3VlQ29tbWVudDE2MjQxNzI4Mw==,7300413,2015-12-07T05:46:15Z,2015-12-07T05:46:15Z,NONE,"I was trying to read ERA-Interim data, calculate anomalies using ds = ds - ds.mean(dim='longitude'), and similar operations along the time axis. Are such operations restricted to single cores? Just multiplying two datasets (u*v) seems to be faster, though top shows two cores being used (I have 4 physical cores). TIA, Joy ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,120681918 https://github.com/pydata/xarray/issues/672#issuecomment-162416658,https://api.github.com/repos/pydata/xarray/issues/672,162416658,MDEyOklzc3VlQ29tbWVudDE2MjQxNjY1OA==,1217238,2015-12-07T05:38:39Z,2015-12-07T05:38:39Z,MEMBER,"What sort of computation are you doing? Some tasks are limited to a single core, notably reading netCDF4 files with in-file compression. Dask's [profiler](http://dask.pydata.org/en/latest/diagnostics.html#profiler) may be helpful here. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,120681918