home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 201617371

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
201617371 MDU6SXNzdWUyMDE2MTczNzE= 1217 Using where() in datasets with dataarrays with different dimensions results in huge RAM consumption 4849151 closed 0     6 2017-01-18T16:09:50Z 2019-02-23T07:47:01Z 2019-02-23T07:47:01Z NONE      

I have a dataset containing groups of data with different dimensions. e.g.:

Python ds = xr.Dataset() ds['data1'] = xr.DataArray(data1, coords={'t1': t1}) ds['data2'] = xr.DataArray(data2, coords={'t2': t2})

If I do something like ds.where(ds.data1 < 0.1), Python ends up allocating huge amounts of memory (>30GB for a dataset of <1MB) and seems to loop indefinitely, until the call is interrupted with CTRL-C.

To use where() successfully, I have to use a subset of the dataset with only one dimension.

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1217/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 6 rows from issue in issue_comments
Powered by Datasette · Queries took 0.853ms · About: xarray-datasette