home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 295270362

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
295270362 MDU6SXNzdWUyOTUyNzAzNjI= 1895 Avoid Adapters in task graphs? 306380 closed 0     13 2018-02-07T19:52:02Z 2022-05-11T20:26:42Z 2022-05-11T20:26:42Z MEMBER      

Looking at an open_zarr computation from @rabernat I'm coming across intermediate values like the following:

```python

Future('zarr-adt-0f90b3f56f247f966e5ef01277f31374').result() ImplicitToExplicitIndexingAdapter(array=LazilyIndexedArray(array=<xarray.backends.zarr.ZarrArrayWrapper object at 0x7fa921fec278>, key=BasicIndexer((slice(None, None, None), slice(None, None, None), slice(None, None, None))))) ```

This object has many dependents, and so will presumably have to float around the network to all of the workers

```python

len(dep.dependents) 1781 ```

In principle this is fine, especially if this object is cheap to serialize, move, and deserialize. It does introduce a bit of friction though. I'm curious how hard it would be to build task graphs that generated these objects on the fly, or else removed them altogether. It is slightly more convenient from a task scheduling perspective for data access tasks to not have any dependencies.

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1895/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 13 rows from issue in issue_comments
Powered by Datasette · Queries took 0.676ms · About: xarray-datasette