html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/731#issuecomment-192107773,https://api.github.com/repos/pydata/xarray/issues/731,192107773,MDEyOklzc3VlQ29tbWVudDE5MjEwNzc3Mw==,15167171,2016-03-04T05:09:54Z,2016-03-04T05:09:54Z,NONE,"Alright, I think it's ready!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/pull/731#issuecomment-189476736,https://api.github.com/repos/pydata/xarray/issues/731,189476736,MDEyOklzc3VlQ29tbWVudDE4OTQ3NjczNg==,15167171,2016-02-26T20:44:05Z,2016-02-26T20:44:05Z,NONE,"Ready for review.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/pull/731#issuecomment-183784767,https://api.github.com/repos/pydata/xarray/issues/731,183784767,MDEyOklzc3VlQ29tbWVudDE4Mzc4NDc2Nw==,15167171,2016-02-14T01:15:49Z,2016-02-14T01:15:49Z,NONE,"Alright I think I got rebase to work, apologies for the delay.
There were some comments I wasn't sure about how to integrate, I asked about them, but I figured the best way to keep moving forward was to push my best guess at what you meant. So we might need to go through one more round of comments. Thanks for being patient!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/pull/731#issuecomment-178379984,https://api.github.com/repos/pydata/xarray/issues/731,178379984,MDEyOklzc3VlQ29tbWVudDE3ODM3OTk4NA==,15167171,2016-02-02T05:12:33Z,2016-02-02T05:12:33Z,NONE,"Thanks! Good comments, should help with my next pull request. : )
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/issues/735#issuecomment-177039255,https://api.github.com/repos/pydata/xarray/issues/735,177039255,MDEyOklzc3VlQ29tbWVudDE3NzAzOTI1NQ==,15167171,2016-01-30T01:18:08Z,2016-01-30T01:26:34Z,NONE,"Note I mis-typed `vecnorm` instead of `vnorm` previously.
> I would be OK adding a norm method, although I don't think there's a super strong need for it -- usually I've been happy writing expressions like (x *\* 2).sum(['shapes', 'x']) *\* 0.5 instead.
When using large dask arrays that operation tended to fill up ram. I'm not sure why, but it made the dask objects very large, I don't have a good understanding of how those graphs are constructed so I was unable to track down the problem.
I tried out `da.std(['shapes', 'x'])` and it worked like a charm, but it felt a little silly rescaling it to get the norm.
> You would either need to implement this all in xarray, or preferably write da.linalg.norm in dask and use that take. Take a look at the scipy source code for this function -- I suspect you could port this almost directly to dask.
`dask` has a `vnorm`,
http://dask.pydata.org/en/latest/array-api.html?highlight=norm#dask.array.core.Array.vnorm
which I am currently trying to make use of, using the very helpful:
`ops._dask_or_eager_func('vnorm', n_array_args=1)`
unfortunately numpy calls their norm `norm` as opposed to `vnorm` so I think I'll need to put in a switch depending on the array type before hitting `_dask_or_eager_func`
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129919128
https://github.com/pydata/xarray/pull/731#issuecomment-176955192,https://api.github.com/repos/pydata/xarray/issues/731,176955192,MDEyOklzc3VlQ29tbWVudDE3Njk1NTE5Mg==,15167171,2016-01-29T20:29:15Z,2016-01-29T20:29:15Z,NONE,"> That said, this is difficult to implement with numpy's dot/tensordot, so perhaps it's better to simply error or omit the dims argument entirely for now. Eventually, we might be able to do this using @/np.matmul (only in numpy 1.10 and newer).
Yeah I'm going to go for omitting the dims entirely for now, I think it makes the function easy to call, and really covers what I imagine most people would use the function for, getting the relationship between two DataArrays along their common dimensions. Thats all I'm going to use it for.
> I also wonder if perhaps we should rename this from tensordot to simply dot. I don't think we would want to use dot for anything else, and it might also be nice to support @ syntax as an alias for this in Python 3 (again, at some later point).
Yeah that makes sense. `tensordot` in xarray seems like it would be the function that allows more flexibility in how a sum product is applied, `dot` can be thought of as a reduced version of `tensordot` taking no `dims` argument.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/pull/731#issuecomment-176571428,https://api.github.com/repos/pydata/xarray/issues/731,176571428,MDEyOklzc3VlQ29tbWVudDE3NjU3MTQyOA==,15167171,2016-01-29T04:44:25Z,2016-01-29T04:45:45Z,NONE,"I've implemented all your comments, except:
1. In the dask test I'm having a little trouble with:
`self.assertLazyAndAllClose(eager, lazy)`
working with the DataArrays I make. Does it only take transformations of:
```
self.eager_array
self.lazy_array
```
2 Your comment:
`Use the genetic type(self) instead`
Which I didn't understand.
3 Need to figure out how to edit the docs.
And I was thinking:
The default (when no dims arg is given) should be to tensordot over all shared dims between DataArrays.
And it should prevent you from not summing over all shared dims, since that will return a DataArray with repeated labels. I think it would be rare for people to want to tensor dot over different dimensions.
Thanks for all your help getting me going!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/pull/731#issuecomment-175905067,https://api.github.com/repos/pydata/xarray/issues/731,175905067,MDEyOklzc3VlQ29tbWVudDE3NTkwNTA2Nw==,15167171,2016-01-27T23:07:37Z,2016-01-27T23:07:37Z,NONE,"It seems in both unsuccesful cases it was trying to import dask and failed, I can't quite figure out what I could have changed to prevent dask from importing:
```
> da = da.chunk()
xarray/test/test_dataarray.py:1618:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
xarray/core/dataarray.py:569: in chunk
ds = self._to_temp_dataset().chunk(chunks)
```
```
try:
from dask.base import tokenize
except ImportError:
> import dask # raise the usual error if dask is entirely missing
E ImportError: No module named 'dask'
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,129286220
https://github.com/pydata/xarray/issues/723#issuecomment-175378477,https://api.github.com/repos/pydata/xarray/issues/723,175378477,MDEyOklzc3VlQ29tbWVudDE3NTM3ODQ3Nw==,15167171,2016-01-27T03:59:46Z,2016-01-27T04:00:24Z,NONE,"Also that einsum does seem pretty ideal. I'll see if I can get it running in dask, so we can port it over here.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,128687346
https://github.com/pydata/xarray/issues/723#issuecomment-175378292,https://api.github.com/repos/pydata/xarray/issues/723,175378292,MDEyOklzc3VlQ29tbWVudDE3NTM3ODI5Mg==,15167171,2016-01-27T03:58:31Z,2016-01-27T03:58:31Z,NONE,"I wasn't sure where the best place to put the def would be. Currently I have been running it from the xarray class:
`t = da1.tensordot( da2, 'shapes' )`
Let me know if that seems alright, then I'll write some simple tests in test_dataarray for tensor dot.
Maybe make my first pull request!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,128687346
https://github.com/pydata/xarray/issues/723#issuecomment-175175494,https://api.github.com/repos/pydata/xarray/issues/723,175175494,MDEyOklzc3VlQ29tbWVudDE3NTE3NTQ5NA==,15167171,2016-01-26T18:53:02Z,2016-01-26T18:53:02Z,NONE,"Looks like it can perform tensor dot for dask and straight xarrays! But apparently dask has not implemented tensordot with multiple axes arguments, and it also does not work performing a tensor dot between a dask xarray and an xarray. Neither of these cases worries me too much, hopefully they don't worry you.
``` python
from xarray import align, DataArray
#note: using private imports (e.g., from xarray.core) is definitely discouraged!
#this is not guaranteed to work in future versions of xarray
from xarray.core.ops import _dask_or_eager_func
def tensordot(a, b, dims):
if not (isinstance(a, DataArray) and isinstance(b, DataArray)):
raise ValueError
a, b = align(a, b, join='inner', copy=False)
axes = (a.get_axis_num(dims), b.get_axis_num(dims))
f = _dask_or_eager_func('tensordot', n_array_args=2)
new_data = f(a.data, b.data, axes=axes)
if isinstance(dims, str):
dims = [dims]
new_coords = a.coords.merge(b.coords).drop(dims)
#drop the dims you are performing the sum product over
new_dims = ([d for d in a.dims if d not in dims] +
[d for d in b.dims if d not in dims])
return DataArray(new_data, new_coords, new_dims)
import xarray as xr
import numpy as np
x_trans = np.linspace(-3,3,6)
y_trans = np.linspace(-3,3,5)
imgID = range(4)
da = xr.DataArray( np.ones((6,5,4)),
coords = [ x_trans, y_trans, imgID ],
dims = ['x_trans', 'y_trans', 'imgID'] )
models = range(20)
dm = xr.DataArray( np.ones(( 20 , 5, 4 )),
coords = [ models, y_trans, imgID],
dims = [ 'models', 'y_trans', 'imgID' ] )
#xarray tensordot
proj_a = tensordot(da, dm, 'imgID')
#dask xarray tensor dot
da = da.chunk()
dm = dm.chunk()
proj_b = tensordot(da, dm, 'imgID')
#errors
#multiple dims
proj_c = tensordot(da, dm, ['imgID', 'y_trans'])
#mixed types
da = da.chunk()
dm = dm.load()
proj_d = tensordot(da, dm, 'imgID')
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,128687346