issue_comments: 435688958

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/pull/2538#issuecomment-435688958 https://api.github.com/repos/pydata/xarray/issues/2538 435688958 MDEyOklzc3VlQ29tbWVudDQzNTY4ODk1OA== 1217238 2018-11-04T17:29:11Z 2018-11-04T17:29:11Z MEMBER

OK, that seems reasonable. The default behavior should cache the arrays loaded with NumPy anyways. I would not be opposed to renaming this to open_dataset, either. On Sun, Nov 4, 2018 at 9:19 AM Joe Hamman notifications@github.com wrote:

@shoyer https://github.com/shoyer - absolutely we'll get better performance with numpy arrays in this case. So I'm trying to use our tutorial datasets for some examples with dask (dask/dask-examples#51 https://github.com/dask/dask-examples/pull/51). The docstring for the load_dataset function states that we can pass kwargs on to the open_dataset function but if we pass chunks to the load_dataset call currently, we still get data back as numpy arrays. We have some other options here:

  1. if chunks is a kwargs, return a dataset with data as persisted dask arrays
  2. provide a second function to handle returning datasets using the same logic as open_dataset (caching, dask arrays, lazy loading, etc.)
  3. tell people (like me) to rechunk the dataset after the fact

(3) won't require any changes but makes it a little harder to connect the typical use pattern of open_dataset with tutorial.load_dataset.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/2538#issuecomment-435688104, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1mBjbk7l2qXi4EqFtMGdvDDoPJHaks5uryGUgaJpZM4YM5-d . 

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   377075253