home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

6 rows where author_association = "MEMBER" and issue = 252548859 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 4

  • shoyer 3
  • rabernat 1
  • jhamman 1
  • crusaderky 1

issue 1

  • (trivial) xarray.quantile silently resolves dask arrays · 6 ✖

author_association 1

  • MEMBER · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
514002753 https://github.com/pydata/xarray/issues/1524#issuecomment-514002753 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDUxNDAwMjc1Mw== shoyer 1217238 2019-07-23T00:18:05Z 2019-07-23T00:18:05Z MEMBER

@shoyer does dask/dask#4677 solve those accuracy concerns?

Yes, to some degree. I'm still troubled by that the "default" algorithm (which is selected by default) has no error bounds. It seems a little backwards to me to default to a fast algorithm with unknown accuracy.

Also, it still only works on 1D arrays, which would not be terribly useful for us.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859
404733610 https://github.com/pydata/xarray/issues/1524#issuecomment-404733610 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDQwNDczMzYxMA== shoyer 1217238 2018-07-13T05:55:14Z 2018-07-13T05:55:14Z MEMBER

See also https://github.com/dask/dask/issues/3099

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859
404733398 https://github.com/pydata/xarray/issues/1524#issuecomment-404733398 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDQwNDczMzM5OA== shoyer 1217238 2018-07-13T05:53:58Z 2018-07-13T05:53:58Z MEMBER

@acrosby if you're at SciPy, I'd be happy to chat about this tomorrow or over the weekend if you're staying for the sprints. This is not an immediate priority for me, but it would be straightforward to make quantile work over non-chunked dimensions by rewriting it to use apply_ufunc.

Approximate quantiles over chunked dimensions could be done by leveraging dask.array.percentile, but that algorithm has some accuracy concerns. See https://github.com/dask/dask/issues/1225 for discussion.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859
325252313 https://github.com/pydata/xarray/issues/1524#issuecomment-325252313 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDMyNTI1MjMxMw== jhamman 2443309 2017-08-28T03:33:39Z 2017-08-28T03:33:39Z MEMBER

@crusaderky - thanks for this report. I just opened #1529 which takes care of the trivial part of this issue. If you want to tackle bringing dask.percentile in, that would be awesome.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859
324616689 https://github.com/pydata/xarray/issues/1524#issuecomment-324616689 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDMyNDYxNjY4OQ== crusaderky 6213168 2017-08-24T12:07:53Z 2017-08-24T12:07:53Z MEMBER

Dask only supports 1d. One would first need to expand dask to support N-dimensional arrays like numpy does. I plan to di it if/when I have the time

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859
324607199 https://github.com/pydata/xarray/issues/1524#issuecomment-324607199 https://api.github.com/repos/pydata/xarray/issues/1524 MDEyOklzc3VlQ29tbWVudDMyNDYwNzE5OQ== rabernat 1197350 2017-08-24T11:18:34Z 2017-08-24T11:18:34Z MEMBER

Dask implements percentile now http://dask.pydata.org/en/latest/array-api.html#dask.array.percentile

So perhaps our version of quantile can be refactored to accommodate actual lazy computation on dask arrays, rather than simply erroring.

In any case, I agree that automatic silent eager evaluation of dask arrays is bad.

Sent from my iPhone

On Aug 24, 2017, at 11:54 AM, crusaderky notifications@github.com wrote:

In variable.py, line 1116, you're missing a raise statement:

    if isinstance(self.data, dask_array_type):
        TypeError("quantile does not work for arrays stored as dask "
                  "arrays. Load the data via .compute() or .load() prior "
                  "to calling this method.")

Currently looking into extending dask.percentile() to support more than 1D arrays, and then use it in xarray too.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  (trivial) xarray.quantile silently resolves dask arrays 252548859

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 1682.235ms · About: xarray-datasette