issue_comments: 286779750

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/1308#issuecomment-286779750 https://api.github.com/repos/pydata/xarray/issues/1308 286779750 MDEyOklzc3VlQ29tbWVudDI4Njc3OTc1MA== 7300413 2017-03-15T15:32:33Z 2017-03-15T15:32:33Z NONE

Not sure if this helps, but I did a %%timeit on both versions. For daily climatology, the numbers are: CPU times: user 1h 21min 8s, sys: 6h 17min 39s, total: 7h 38min 47s Wall time: 20min 34s

For the 6 hourly thing, CPU times: user 5h 5min 6s, sys: 1d 2h 19min 45s, total: 1d 7h 24min 51s Wall time: 1h 31min 40s

It takes around 4x more time, which makes sense because there are 4x more groups. The ratio of user to system time is more or less constant, so nothing untoward seems to be happening in between the two runs.

I think it is just good to remember that the time to use scales linearly with the number of groups. I guess this is what @shoyer was talking about when he mentioned that since grouping is done within xarray, the dask graph grows, making things slower.

Thanks again!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   214088387