issue_comments

Table actions

3 rows where issue = 550355524 and user = 1312546 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • TomAugspurger · 3 ✖

issue 1

  • dask.optimize on xarray objects · 3 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
690378323 https://github.com/pydata/xarray/issues/3698#issuecomment-690378323 https://api.github.com/repos/pydata/xarray/issues/3698 MDEyOklzc3VlQ29tbWVudDY5MDM3ODMyMw== TomAugspurger 1312546 2020-09-10T15:42:54Z 2020-09-10T15:42:54Z MEMBER

Thanks for confirming. I'll take another look at this today then.

On Thu, Sep 10, 2020 at 10:30 AM Deepak Cherian notifications@github.com wrote:

Reopened #3698 https://github.com/pydata/xarray/issues/3698.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3698#event-3751728444, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIT6LDBKVUQ5KR7VFB3SFDWI3ANCNFSM4KHH63GQ . 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.optimize on xarray objects 550355524
689808725 https://github.com/pydata/xarray/issues/3698#issuecomment-689808725 https://api.github.com/repos/pydata/xarray/issues/3698 MDEyOklzc3VlQ29tbWVudDY4OTgwODcyNQ== TomAugspurger 1312546 2020-09-09T20:38:39Z 2020-09-09T20:38:39Z MEMBER

FYI, @dcherian your recent PR to dask fixed this example. Playing around with chunk sizes, it seems to have fixed it even when the chunk size exceeds dask.config['array']['chunk-size'].

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.optimize on xarray objects 550355524
592101136 https://github.com/pydata/xarray/issues/3698#issuecomment-592101136 https://api.github.com/repos/pydata/xarray/issues/3698 MDEyOklzc3VlQ29tbWVudDU5MjEwMTEzNg== TomAugspurger 1312546 2020-02-27T18:13:28Z 2020-02-27T18:13:28Z MEMBER

It looks like xarray is getting a bad task graph after the optimize.

```python In [1]: import xarray as xr import dask In [2]: import dask

In [3]: a = dask.array.ones((10,5), chunks=(1,3)) ...: a = dask.optimize(a)[0]

In [4]: da = xr.DataArray(a.compute()).chunk({"dim_0": 5}) ...: da = dask.optimize(da)[0]

In [5]: dict(da.dask_graph()) Out[5]: {('xarray-<this-array>-e2865aa10d476e027154771611541f99', 1, 0): (<function _operator.getitem(a, b, /)>, 'xarray-<this-array>-e2865aa10d476e027154771611541f99', (slice(5, 10, None), slice(0, 5, None))), ('xarray-<this-array>-e2865aa10d476e027154771611541f99', 0, 0): (<function _operator.getitem(a, b, /)>, 'xarray-<this-array>-e2865aa10d476e027154771611541f99', (slice(0, 5, None), slice(0, 5, None)))} ```

Notice that are references to xarray-<this-array>-e2865aa10d476e027154771611541f99 (just the string, not a tuple representing a chunk) but that key isn't in the graph.

If we manually insert that, you'll see things work

```python In [9]: dsk['xarray-<this-array>-e2865aa10d476e027154771611541f99'] = da._to_temp_dataset()[xr.core.dataarray._THIS_ARRAY]

In [11]: dask.get(dsk, keys=[('xarray-<this-array>-e2865aa10d476e027154771611541f99', 1, 0)]) Out[11]: (<xarray.DataArray \<this-array> (dim_0: 5, dim_1: 5)> dask.array<getitem, shape=(5, 5), dtype=float64, chunksize=(5, 5), chunktype=numpy.ndarray> Dimensions without coordinates: dim_0, dim_1,) ```

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   dask.optimize on xarray objects 550355524

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);