home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 521758170

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2808#issuecomment-521758170 https://api.github.com/repos/pydata/xarray/issues/2808 521758170 MDEyOklzc3VlQ29tbWVudDUyMTc1ODE3MA== 22258697 2019-08-15T19:02:51Z 2021-07-21T16:47:47Z NONE

Ryan Abernathey gave a helpful answer for how to apply a pixel wise function using dask and apply_ufunc: https://stackoverflow.com/questions/57419541/how-to-use-apply-ufunc-with-numpy-digitize-for-each-image-along-time-dimension-o/57513184#57513184

I think the docs could improve on showing how to use apply_ufunc if we have a function that needs to be applied image-wise, like an image filter or segmentation, if we are chunking by time. Or, if the function needs to be applied window-wise, in which case the chunks are spatial (maybe DataArray.rolling and DataArray.reduce solve this case, but DataArray.reduce lacks an example).

Having examples that speak to these 2 specific use cases would, I think, help newcomers (like myself) that are coming from any domain that works with 2D ('x', 'y') or 3D ('x', 'y', 'time') arrays.

Currently the two examples in the docs show how to apply_ufunc with a 1D array http://xarray.pydata.org/en/stable/computation.html#comput-wrapping-custom

And two 2D arrays ('place', 'time') http://xarray.pydata.org/en/stable/dask.html#automatic-parallelization

Some other comments on my, and possibly others', points of confusion.

  1. I'm not sure what a gufunc is, and if this is different than a ufunc (see the spearman_correlation function)
  2. After rereading both pages and numpy docs to understand universal functions, I have some intuition about what input_core_dims does, but I still don't have a great enough understanding to know how to use apply_ufunc to operate across 3D arrays that are chunked by time or space.
  3. The api reference for apply_ufunc renders such that some arg names have no whitespace between the arg type. http://xarray.pydata.org/en/stable/generated/xarray.apply_ufunc.html

  4. apply_ufunc seems to have the flexibility to support operations that output DataArrays of reduced shape, with arguments named like output_core_dims and exclude_dims. However, I tried to use it with a custom function that takes as input a single 3D image ('x', 'y', 'band') in my time series and returns a tuple of an intercept and slope computed from regressing the blue and red bands of that image. I tried various arguments but kept running into errors. I think an example that shows how to use apply_ufunc where the output has a different, reduced shape than any of the inputs would be valuable.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  420584430
Powered by Datasette · Queries took 0.867ms · About: xarray-datasette