home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 634497758

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/pull/4060#issuecomment-634497758 https://api.github.com/repos/pydata/xarray/issues/4060 634497758 MDEyOklzc3VlQ29tbWVudDYzNDQ5Nzc1OA== 5821660 2020-05-27T08:02:24Z 2020-05-27T08:02:24Z MEMBER

@mathause Thanks for the elaborate review. But, I'm unsure how to move on with this.

I'll reply to some of your comments below.

Note that output_dtypes is used in vectorize so this may not be a candidate for dask_gufunc_kwargs.

Hmm, the docstring indicates that it is only used with dask='parallelized'. But in the code it can be used in any case. This should be clarified.

The second is not so trivial. I see three possibilities (1) just error, (2) try dask.array.apply_gufunc if that fails issue a warning and use the old apply_blockwise (3) figure out ourselves if non-core dimensions (called loop dimensions in dask) are not-equally chunked, issue a warning and re-chunk them ourselves. Maybe @shoyer and @dcherian can weight in here.

When looking at the code of dask.array.apply_gufunc it seems, that any possible re-chunking is done in dask.array.blockwise. I also do not see any special treatment of warning or re-chunking of non-core dimensions in the current _apply_blockwise. So from my reasoning we should be safe to use (4) try dask.array.apply_gufunc since dask.array.apply_gufunc will issue the appropriate ValueError if something is wrong with the chunking. (Only thing is that the ValueError concerning the non-core dimension doesn't mention allow_rechunk). If the ValueError is issued, the user can the just set allow_rechunk=True keyword.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  618081836
Powered by Datasette · Queries took 244.176ms · About: xarray-datasette