issue_comments: 415450958
This data as json
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | performed_via_github_app | issue |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/pydata/xarray/issues/2329#issuecomment-415450958 | https://api.github.com/repos/pydata/xarray/issues/2329 | 415450958 | MDEyOklzc3VlQ29tbWVudDQxNTQ1MDk1OA== | 12278765 | 2018-08-23T15:02:57Z | 2018-08-23T15:02:57Z | NONE | It seems that I managed to get something working as it should.
I first load my monthly grib files with iris, convert to xarray, then write to zarr. This uses all the CPU cores, but loads the full array into memory. Since the individual arrays are relatively small, that is not an issue.
Then I load the monthly zarr stores with xarray, concatenate them with Using that big zarr storage, plotting a map of the mean values along the time axis takes around 15min, uses all the cores and around 24GB of RAM. The strange part is: I think I tried that before and it was not working... |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
345715825 |