home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

8 rows where issue = 128687346 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 3

  • shoyer 4
  • deanpospisil 3
  • max-sixty 1

author_association 2

  • MEMBER 5
  • NONE 3

issue 1

  • Implement tensordot for xarray with dask support · 8 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
192539434 https://github.com/pydata/xarray/issues/723#issuecomment-192539434 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE5MjUzOTQzNA== shoyer 1217238 2016-03-05T00:47:49Z 2016-03-05T00:47:49Z MEMBER

Fixed by #731.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
175383509 https://github.com/pydata/xarray/issues/723#issuecomment-175383509 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NTM4MzUwOQ== shoyer 1217238 2016-01-27T04:24:42Z 2016-01-27T04:24:42Z MEMBER

I'm split on whether a function or method makes more sense (a.tensordot(b, dim='x') vs xr.tensordot(a, b, dim='x')). I would be OK with either, so yes, please do go ahead!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
175378477 https://github.com/pydata/xarray/issues/723#issuecomment-175378477 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NTM3ODQ3Nw== deanpospisil 15167171 2016-01-27T03:59:46Z 2016-01-27T04:00:24Z NONE

Also that einsum does seem pretty ideal. I'll see if I can get it running in dask, so we can port it over here.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
175378292 https://github.com/pydata/xarray/issues/723#issuecomment-175378292 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NTM3ODI5Mg== deanpospisil 15167171 2016-01-27T03:58:31Z 2016-01-27T03:58:31Z NONE

I wasn't sure where the best place to put the def would be. Currently I have been running it from the xarray class: t = da1.tensordot( da2, 'shapes' ) Let me know if that seems alright, then I'll write some simple tests in test_dataarray for tensor dot. Maybe make my first pull request!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
175175494 https://github.com/pydata/xarray/issues/723#issuecomment-175175494 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NTE3NTQ5NA== deanpospisil 15167171 2016-01-26T18:53:02Z 2016-01-26T18:53:02Z NONE

Looks like it can perform tensor dot for dask and straight xarrays! But apparently dask has not implemented tensordot with multiple axes arguments, and it also does not work performing a tensor dot between a dask xarray and an xarray. Neither of these cases worries me too much, hopefully they don't worry you.

``` python from xarray import align, DataArray

note: using private imports (e.g., from xarray.core) is definitely discouraged!

this is not guaranteed to work in future versions of xarray

from xarray.core.ops import _dask_or_eager_func

def tensordot(a, b, dims): if not (isinstance(a, DataArray) and isinstance(b, DataArray)): raise ValueError

a, b = align(a, b, join='inner', copy=False)

axes = (a.get_axis_num(dims), b.get_axis_num(dims))
f = _dask_or_eager_func('tensordot', n_array_args=2)
new_data = f(a.data, b.data, axes=axes)

if isinstance(dims, str):
    dims = [dims]

new_coords = a.coords.merge(b.coords).drop(dims)

#drop the dims you are performing the sum product over
new_dims = ([d for d in a.dims if d not in dims] +
            [d for d in b.dims if d not in dims])

return DataArray(new_data, new_coords, new_dims)

import xarray as xr import numpy as np

x_trans = np.linspace(-3,3,6) y_trans = np.linspace(-3,3,5) imgID = range(4) da = xr.DataArray( np.ones((6,5,4)), coords = [ x_trans, y_trans, imgID ], dims = ['x_trans', 'y_trans', 'imgID'] )

models = range(20) dm = xr.DataArray( np.ones(( 20 , 5, 4 )), coords = [ models, y_trans, imgID], dims = [ 'models', 'y_trans', 'imgID' ] )

xarray tensordot

proj_a = tensordot(da, dm, 'imgID')

dask xarray tensor dot

da = da.chunk() dm = dm.chunk() proj_b = tensordot(da, dm, 'imgID')

errors

multiple dims

proj_c = tensordot(da, dm, ['imgID', 'y_trans'])

mixed types

da = da.chunk() dm = dm.load() proj_d = tensordot(da, dm, 'imgID') ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
174841649 https://github.com/pydata/xarray/issues/723#issuecomment-174841649 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NDg0MTY0OQ== max-sixty 5635139 2016-01-26T05:37:47Z 2016-01-26T05:37:47Z MEMBER

@shoyer - I thought your answer dominated mine, so I left yours as the only response. But yup, that form of einsum would be pretty nice...

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
174841301 https://github.com/pydata/xarray/issues/723#issuecomment-174841301 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NDg0MTMwMQ== shoyer 1217238 2016-01-26T05:34:40Z 2016-01-26T05:34:40Z MEMBER

@MaximilianR I do like einsum, but I'm not sure the API would be a good fit for xarray (we already have dimension names), and it also does not exist yet for dask (https://github.com/blaze/dask/issues/732).

That said, I suppose you could make an xarray version of einsum with syntax that looks more like tensordot with *args, e.g., einsum(a, b, c, dims=('x', 'y')).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346
174839547 https://github.com/pydata/xarray/issues/723#issuecomment-174839547 https://api.github.com/repos/pydata/xarray/issues/723 MDEyOklzc3VlQ29tbWVudDE3NDgzOTU0Nw== shoyer 1217238 2016-01-26T05:27:36Z 2016-01-26T05:32:02Z MEMBER

Yes, this would be a nice addition!

I spent a little bit of a time futzing around with this to see if there is an elegant way to plug this into our existing dispatching system. The short of it is that the answer appears to be no -- we don't have any elegant equivalent to dask.array's generic atop method.

So, for now I would simply write a function specialized to DataArray objects. Something like the following (barely tested) is a starting point:

``` python from xarray import align, DataArray

note: using private imports (e.g., from xarray.core) is definitely discouraged!

this is not guaranteed to work in future versions of xarray

from xarray.core.ops import _dask_or_eager_func

def tensordot(a, b, dims): if not (isinstance(a, DataArray) and isinstance(b, DataArray)): raise ValueError

a, b = align(a, b, join='inner', copy=False)

axes = (a.get_axis_num(dims), b.get_axis_num(dims))
f = _dask_or_eager_func('tensordot', n_array_args=2)
new_data = f(a.data, b.data, axes=axes)

if isinstance(dims, basestring):
    dims = [dims]

new_coords = a.coords.merge(b.coords).drop(dims)

new_dims = ([d for d in a.dims if d not in dims] +
            [d for d in b.dims if d not in dims])

return DataArray(new_data, new_coords, new_dims)

```

This would be worth cleaning up so we could add it to the codebase (mostly documentation & tests).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Implement tensordot for xarray with dask support 128687346

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 13.784ms · About: xarray-datasette