html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/912#issuecomment-234056046,https://api.github.com/repos/pydata/xarray/issues/912,234056046,MDEyOklzc3VlQ29tbWVudDIzNDA1NjA0Ng==,1217238,2016-07-20T19:29:55Z,2016-07-20T19:29:55Z,MEMBER,"Just looking at a task manager while a task executes can give you a sense
of what's going on. Dask also has some diagnostics that may be helpful:
http://dask.pydata.org/en/latest/diagnostics.html
On Wed, Jul 20, 2016 at 11:44 AM Saulo Meirelles notifications@github.com
wrote:

> No, not really. I got no error message whatsoever. Is there any test I can
> do to tackle this?
> 
> Sent from Smartphone. Please forgive typos.
> 
> On Jul 20, 2016 8:41 PM, ""Stephan Hoyer"" notifications@github.com wrote:
> 
> > I decided to wait for .load() to do the job but the kernel dies after a
> > while.
> > 
> > Are you running out of memory? Can you tell what's going on? This is a
> > little surprising to me.
> > 
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> > https://github.com/pydata/xarray/issues/912#issuecomment-234042142,
> > or mute
> > the thread
> > <
> > https://github.com/notifications/unsubscribe-auth/AHKCTXaBpbA0ieSdI2I_hIUjVBxuKaNeks5qXmvPgaJpZM4JQ0_D
> > 
> > .
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/xarray/issues/912#issuecomment-234043292, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ABKS1ujXItyYDLgA4ZtBkHEbTBTiTnrvks5qXmylgaJpZM4JQ0_D
> .
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563
https://github.com/pydata/xarray/issues/912#issuecomment-234042142,https://api.github.com/repos/pydata/xarray/issues/912,234042142,MDEyOklzc3VlQ29tbWVudDIzNDA0MjE0Mg==,1217238,2016-07-20T18:41:17Z,2016-07-20T18:41:17Z,MEMBER,"> I decided to wait for .load() to do the job but the kernel dies after a while.

Are you running out of memory? Can you tell what's going on? This is a little surprising to me.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563
https://github.com/pydata/xarray/issues/912#issuecomment-234026185,https://api.github.com/repos/pydata/xarray/issues/912,234026185,MDEyOklzc3VlQ29tbWVudDIzNDAyNjE4NQ==,1217238,2016-07-20T17:47:45Z,2016-07-20T17:47:45Z,MEMBER,"It's worth noting that `conc_avg = ds.conc_profs.chunk({'burst': 10}).mean(('z','duration'))` doesn't actually do any computation -- that's why it's so fast. It just sets up the computation graph. No computation happens until you write `.load()`.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563
https://github.com/pydata/xarray/issues/912#issuecomment-233998757,https://api.github.com/repos/pydata/xarray/issues/912,233998757,MDEyOklzc3VlQ29tbWVudDIzMzk5ODc1Nw==,1217238,2016-07-20T16:11:27Z,2016-07-20T16:11:27Z,MEMBER,"When you write `ds.conc_profs.chunk(2400)`, it sets up the data to be loaded in a giant chunk, almost the entire file at once. Even if you use `.isel()` afterwards, dask does not always manage to subset the data from the initial chunk. (Sometimes it does succeed, which makes this a little confusing.)

You will probably be more successful if you try something like `ds.conc_profs.chunk({'burst': 10})` instead, which keeps the intermediate chunks to a reasonable size.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563
https://github.com/pydata/xarray/issues/912#issuecomment-233996527,https://api.github.com/repos/pydata/xarray/issues/912,233996527,MDEyOklzc3VlQ29tbWVudDIzMzk5NjUyNw==,1217238,2016-07-20T16:03:30Z,2016-07-20T16:03:30Z,MEMBER,"Thanks for describing that -- I misread your initial description and thought you were using `open_mfdataset` rather than `open_dataset` (the glob threw me off!). The source of these files shouldn't matter once you have it in a netCDF file.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563
https://github.com/pydata/xarray/issues/912#issuecomment-233991357,https://api.github.com/repos/pydata/xarray/issues/912,233991357,MDEyOklzc3VlQ29tbWVudDIzMzk5MTM1Nw==,1217238,2016-07-20T15:46:50Z,2016-07-20T15:46:50Z,MEMBER,"What do the original input files look like, before you join them together? This may be a case where the dask.array task scheduler does very poorly.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,166593563