home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 1210383450

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6904#issuecomment-1210383450 https://api.github.com/repos/pydata/xarray/issues/6904 1210383450 IC_kwDOAMm_X85IJPxa 12760310 2022-08-10T09:07:00Z 2022-08-10T09:07:00Z NONE

This is a minimal working example that I could come up with. You can try to open any netcdf that you have. I tested on a small one and it didn't reproduce the error, so it is definitely only happening with large datasets when the arrays are not loaded into memory. Unfortunately, as you need a large file, I cannot really attach it here.

```python import xarray as xr from tqdm.contrib.concurrent import process_map import pprint

def main(): global ds ds = xr.open_dataset('input.nc') it = range(0, 5) results = [] for i in it: results.append(compute(i)) print("------------Serial results-----------------") pprint.pprint(results) results = process_map(compute, it, max_workers=6, chunksize=1, disable=True) print("------------Parallel results-----------------") pprint.pprint(results)

def compute(station): ds_point = ds.isel(lat=0, lon=0) return station, ds_point.t_2m_max.mean().item(), ds_point.t_2m_min.mean().item(), ds_point.lon.min().item(), ds_point.lat.min().item()

if name == "main": main() ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1333650265
Powered by Datasette · Queries took 74.608ms · About: xarray-datasette