issue_comments

Table actions

4 rows where issue = 172290413 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

issue 1

  • broadcast() broken on dask backend · 4 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
250795790 https://github.com/pydata/xarray/issues/978#issuecomment-250795790 https://api.github.com/repos/pydata/xarray/issues/978 MDEyOklzc3VlQ29tbWVudDI1MDc5NTc5MA== crusaderky 6213168 2016-09-30T16:53:23Z 2016-09-30T16:53:23Z MEMBER

Rebaselined #1023 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   broadcast() broken on dask backend 172290413
250778472 https://github.com/pydata/xarray/issues/978#issuecomment-250778472 https://api.github.com/repos/pydata/xarray/issues/978 MDEyOklzc3VlQ29tbWVudDI1MDc3ODQ3Mg== crusaderky 6213168 2016-09-30T15:42:02Z 2016-09-30T15:42:02Z MEMBER

Two-liner for the win #1022 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   broadcast() broken on dask backend 172290413
250770375 https://github.com/pydata/xarray/issues/978#issuecomment-250770375 https://api.github.com/repos/pydata/xarray/issues/978 MDEyOklzc3VlQ29tbWVudDI1MDc3MDM3NQ== crusaderky 6213168 2016-09-30T15:12:04Z 2016-09-30T15:12:04Z MEMBER

looking into this now

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   broadcast() broken on dask backend 172290413
241231815 https://github.com/pydata/xarray/issues/978#issuecomment-241231815 https://api.github.com/repos/pydata/xarray/issues/978 MDEyOklzc3VlQ29tbWVudDI0MTIzMTgxNQ== shoyer 1217238 2016-08-21T00:31:11Z 2016-08-21T00:31:11Z MEMBER

Oops -- let's add a fix for this and a regression test in test_dask.py.

We should fix broadcast as you mention, but also fix the as_compatible_data function to try coercing data via the .data attribute before using .values: https://github.com/pydata/xarray/blob/584e70378c64e3fa861e5b4b4fd61d21639661c6/xarray/core/variable.py#L146

After that however there's a new issue: whenever broadcast adds a dimension to an array, it creates it in a single chunk, as opposed to copying the chunking of the other arrays. This can easily call a host to go out of memory, and makes it harder to work with the arrays afterwards because chunks won't match.

This is sort of but not completely right. We use dask.array.broadcast_to to expand dimensions for dask arrays, which under the hood uses numpy.broadcast_to for each chunk. Broadcasting uses a view to insert a new dimensions with stride 0, so it doesn't require any additional storage costs for the original array. But any arrays resulting from arithmetic will indeed require more space.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   broadcast() broken on dask backend 172290413

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);