html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1914#issuecomment-435717326,https://api.github.com/repos/pydata/xarray/issues/1914,435717326,MDEyOklzc3VlQ29tbWVudDQzNTcxNzMyNg==,10928117,2018-11-04T23:07:56Z,2018-11-04T23:07:56Z,NONE,"@jcmgray I had to miss your reply to this issue, I saw it just now. I love your code! I will definitely include xyzpy in my tools from now on ;-).","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,297560256
https://github.com/pydata/xarray/issues/1914#issuecomment-367677038,https://api.github.com/repos/pydata/xarray/issues/1914,367677038,MDEyOklzc3VlQ29tbWVudDM2NzY3NzAzOA==,10928117,2018-02-22T13:15:11Z,2018-02-22T13:15:11Z,NONE,"@shoyer Thanks for your suggestions and linking the other issue. I think this one can also be labelled as the ""usage question"".","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,297560256
https://github.com/pydata/xarray/issues/1914#issuecomment-366833780,https://api.github.com/repos/pydata/xarray/issues/1914,366833780,MDEyOklzc3VlQ29tbWVudDM2NjgzMzc4MA==,10928117,2018-02-20T00:27:36Z,2018-02-20T00:27:36Z,NONE,"After preparing list similar to ``[{'x': 0, 'y': 'a'}, {'x': 1, 'y': 'a'}, ...]`` interaction with cluster is quite efficient. One can easily pass such a thing to ``async_map`` of ``ipyparallel``. 

Thanks for your suggestion, I need to try few things. I also want to try to extend it to function that computes few different things that could be multi-valued, e.g.
```python
def dummy(x, y):
    ds = xr.Dataset(
        {'out1': ('n', [1*x, 2*x, 3*x]), 'out2': ('m', [x, y])},
        coords = {'x': x, 'y': y, 'n': range(3), 'm': range(2)}
    )
    
    return ds
```
and then group together such outputs... Ok, I know. I go from simple problem to much more complicated one, but isn't it the case usually?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,297560256
https://github.com/pydata/xarray/issues/1914#issuecomment-366819497,https://api.github.com/repos/pydata/xarray/issues/1914,366819497,MDEyOklzc3VlQ29tbWVudDM2NjgxOTQ5Nw==,10928117,2018-02-19T22:40:17Z,2018-02-19T22:58:02Z,NONE,"For ""get done"" I had for example the following (similar to what I linked as my initial attempt)
```python
coordinates = {
    'x': np.linspace(-1, 1), 
    'y': np.linspace(0, 10), 
}

constants = {
    'a': 1, 
    'b': 5
}

inps = [{**constants, **{k: v for k, v in zip(coordinates.keys(), x)}} 
        for x in list(it.product(*coordinates.values()))]


def f(x, y, a, b):
    """"""Some dummy function.""""""
    v = a * x**2 + b * y**2
    return xr.DataArray(v, {'x': x, 'y': y, 'a': a, 'b': b})


# simulate computation on cluster
values = list(map(lambda s: f(**s), inps))

# gather and unstack the inputs
ds = xr.concat(values, dim='new', coords='all')
ds = ds.set_index(new=list(set(ds.coords) - set(ds.dims)))
ds = ds.unstack('new')
```

It is very close to what you suggest. My main question is if this can be done better. Mainly I am wondering if 
1. Is there any built-in iterator over the Cartesian product of coordinates. If no, are there people that also think it would be useful?
2. Gathering together / unstacking of the data. My 3 line combo of ``concat``, ``set_index`` and ``unstack`` seems to do the trick but it seems a bit like over complication. Ideally I'd expect to have some mechanism that works similar to:

```python
inputs = cartesian_product(coordinates) # list similar to ``inps`` above
values = [function(inp) for inp in inputs]  # or using ipypparallel map

xarray_data = ... # some empty xarray object
for inp, val in zip(inputs, values):
   xarray_data[inp] = val
```

I asked how to generate product of coordinates from xarray object because I was expecting that I can create ``xarray_data`` as an empty object with all coordinates set and then fill it. 

---
### Added comment
Having an empty, as filled with ``nan``s, object to start with would have this benefit that one could save partial results and have clean information what was already computed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,297560256
https://github.com/pydata/xarray/issues/1914#issuecomment-366740505,https://api.github.com/repos/pydata/xarray/issues/1914,366740505,MDEyOklzc3VlQ29tbWVudDM2Njc0MDUwNQ==,10928117,2018-02-19T16:20:15Z,2018-02-19T16:23:31Z,NONE,"Let me give a bit of a background what I would like to do:

1. Create an empty ``Dataset`` of coordinates I want to explore, i.e. two np.arrays ``x`` and ``y``, and two scalars ``a`` and ``b``.
2. Generate an list of the Cartesian product of all the coordinates, i.e. ``[ {'x': -1, 'y': 0, 'a': 1, 'b': 5}, ...]`` (data format doesn't really matter).
3. For each item of the iterator compute some function: ``f = f(x, y, a, b)``. In principle this function can be expensive to compute, therefore I'd compute it for each item of list from 2. separately on the cluster.
4. ""merge"" it all together into a single xarray object 

In principle ``f`` should be allowed to return e.g. ``np.array``. 
An related issue in [holoviews](https://github.com/ioam/holoviews/issues/2341#issuecomment-365925725) and the notebook with my initial [attempt](https://gitlab.kwant-project.org/qt/cookbook/blob/2f03d563342be6f5b85190faac24656457f1647f/xarray_holoviews_gridded.ipynb). 
In the linked notebook I managed to achieve the goal however without starting with an ``xarray`` object containing coordinates. Also combining the data seems a bit inefficient as it takes more time than generating it for a larger datasets.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,297560256