home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 593825520

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
593825520 MDU6SXNzdWU1OTM4MjU1MjA= 3932 Element wise dataArray generation 11750960 closed 0     6 2020-04-04T12:24:16Z 2020-04-07T04:32:12Z 2020-04-07T04:32:12Z CONTRIBUTOR      

I'm in a situation where I want to generate a bidimensional DataArray from a method that takes each of the two dimensions as input parameters. I have two methods to do this but neither of these looks particularly elegant to me and I wondered whether somebody would have better ideas.

  • Method 1 : dask delayed

``` x = np.arange(10) y = np.arange(20)

some_exp = lambda x, y: np.ones((Nstats)) some_exp_delayed = dask.delayed(some_exp, pure=True)

lazy_data = [some_exp_delayed(_x, _y) for _x in x for _y in y] sample = lazy_data[0].compute() arrays = [da.from_delayed(lazy_value, dtype=sample.dtype, shape=sample.shape) for lazy_value in lazy_data]

stack = (da.stack(arrays, axis=0) .reshape((len(x),len(y),sample.size)) ) ds = xr.DataArray(stack, dims=['x','y','stats']) ```

I tend to prefer this option because it imposes less requirement on the some_exp data shape. That being said it still seems like too many lines of code to achieve such result.

  • Method 2: apply_ufunc

``` x = np.arange(10) y = np.arange(20)

ds = xr.Dataset(coords={'x': x, 'y': y}) ds['_y'] = (0*ds.x+ds.y) # breaks apply_ufunc otherwise ds = ds.chunk({'x': 1, 'y':1})

let's say each experiment outputs 5 statistical diagnostics

Nstats = 5 some_exp = lambda x, y: np.ones((1,1,Nstats))

out = (xr.apply_ufunc(some_exp, ds.x, ds._y, dask='parallelized', output_dtypes=[float], output_sizes={'stats': Nstats}, output_core_dims=[['stats']]) ) `` I don't understand why I have to use the dummy variableds._yin this case. Having to rely onapply_ufunc` seems like an overkill.

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3932/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 6 rows from issue in issue_comments
Powered by Datasette · Queries took 1.066ms · About: xarray-datasette