home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

6 rows where author_association = "MEMBER" and issue = 282178751 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

  • mrocklin 2
  • shoyer 2
  • jhamman 2

issue 1

  • Add compute=False keywords to `to_foo` functions · 6 ✖

author_association 1

  • MEMBER · 6 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
367806166 https://github.com/pydata/xarray/issues/1784#issuecomment-367806166 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM2NzgwNjE2Ng== shoyer 1217238 2018-02-22T20:08:46Z 2018-02-22T20:08:46Z MEMBER

sync=False is a private, undocumented API. I think it's only found on the internal to_netcdf function in xarray.backends.api, not the to_netcdf method.

Currently, to_netcdf() will either return: - None if a path or file-like object was passed to to_netcdf(). - bytes representing the file's data if no path or file was provided.

In either case I think it could make sense to optionally replace the return value with a dask delayed object, which in turn evaluates to either None or bytes when computed.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751
367166682 https://github.com/pydata/xarray/issues/1784#issuecomment-367166682 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM2NzE2NjY4Mg== jhamman 2443309 2018-02-21T00:10:04Z 2018-02-21T00:10:04Z MEMBER

What does ds.to_netcdf(...) usually return?

If sync == False the store is returned, otherwise nothing is returned.

The term future, when used in a Dask context, generally refers to something that is off computing asynchronously somewhere, rather than a token that holds onto a yet-to-be-submitted lazy graph.

Thanks for the clarification. I wasn't aware of that distinction but it does make sense.

What is store in this case?

A store is AbstractWritableDataStore, basically a wrapper class to allow us to read/write to various fileformats with various APIs under a common interface. Notably, each store has a writer attribute with a sync method that calls dask.array.store.


Another way to do this would be to have user code interact with the sync method directly:

```Python store = ds.to_netcdf('file.nc', sync=False)

store.sync calls store.writer.sync() which calls dask.array.sync

delayed_things = store.sync(compute=False) ```

This has the advantage of keeping the to_netcdf method a bit cleaner but does expose the AbstractWritableDataStore to user code which is typically not a public API object.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751
367163589 https://github.com/pydata/xarray/issues/1784#issuecomment-367163589 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM2NzE2MzU4OQ== mrocklin 306380 2018-02-20T23:55:25Z 2018-02-20T23:55:25Z MEMBER

The term future, when used in a Dask context, generally refers to something that is off computing asynchronously somewhere, rather than a token that holds onto a yet-to-be-submitted lazy graph.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751
367163456 https://github.com/pydata/xarray/issues/1784#issuecomment-367163456 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM2NzE2MzQ1Ng== mrocklin 306380 2018-02-20T23:54:46Z 2018-02-20T23:54:46Z MEMBER

What does ds.to_netcdf(...) usually return? Typically when we specify compute=False we usually return a dask-like thing, often a dask.delayed object that evaluates to what would have been computed normally.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751
367162456 https://github.com/pydata/xarray/issues/1784#issuecomment-367162456 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM2NzE2MjQ1Ng== jhamman 2443309 2018-02-20T23:49:41Z 2018-02-20T23:49:41Z MEMBER

@shoyer - Do you have thoughts on how this feature would present to the user? In #1811, I have added the compute keyword argument to to_netcdf and to_zarr and put a futures attribute on each store. So the workflow there would be something like:

```Python store = ds.to_netcdf('file.nc', compute=False)

dask.compute(store.futures) ```

Before I spend too much time on #1811, I want to get some buy in on the API for this feature.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751
351785902 https://github.com/pydata/xarray/issues/1784#issuecomment-351785902 https://api.github.com/repos/pydata/xarray/issues/1784 MDEyOklzc3VlQ29tbWVudDM1MTc4NTkwMg== shoyer 1217238 2017-12-14T17:48:48Z 2017-12-14T17:48:48Z MEMBER

Yes, this sounds like a natural improvement to me!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Add compute=False keywords to `to_foo` functions 282178751

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 14.555ms · About: xarray-datasette