issue_comments

On Apr 21 2018 10:17 PM, Keisuke Fujii wrote:

How about reset_index? python array.stack(z=('x', 'y')).reset_index('z')

Before I left work for the weekend I had tried array.stack(z=('x', 'y')), but I had not come across reset_index yet. I will give that a try ASAP.

EBo --

Thank you rabernat . I just tried:

``` array.stack(z=('x', 'y')) X

<xarray.DataArray (band: 3, z: 1647861)> dask.array<shape=(3, 1647861), dtype=float64, chunksize=(1, 1647861)> Coordinates: * band (band) int64 1 2 3 * z (z) MultiIndex - x (z) float64 1.939e+05 1.939e+05 1.939e+05 1.939e+05 1.939e+05 ... - y (z) float64 4.986e+06 4.985e+06 4.985e+06 4.984e+06 4.984e+06 ... nX = client.compute(X) ``` and got the following error:

/home/jldavid3/anaconda3/envs/pangeo/lib/python3.6/site-packages/distributed/worker.py:741: UserWarning: Large object of size 6.61 MB detected in task graph: ([[["('reshape-33c73e5277bff381fea27bc752d60c16', ... e, None), None) Consider scattering large objects ahead of time with client.scatter to reduce scheduler burden and keep data on workers

future = client.submit(func, big_data)    # bad

big_future = client.scatter(big_data)     # good
future = client.submit(func, big_future)  # good

% (format_bytes(len(b)), s)) distributed.worker - WARNING - Compute Failed Function: _dask_finalize args: ([[[array([[ 1.11333953, 0.15302669, 2.30724196, ..., -0.49583333, -0.31415252, 0.17898109]])], [array([[ 0.2049355 , 1.32097473, -1.11873895, ..., -0.10651731, 0.69806911, 1.34692913]])], [array([[ 0.59425151, -0.52178773, 0.80188672, ..., -0.83324054, -0.54774213, -0.15842612]])]]], <function Dataset._dask_postcompute at 0x7fbaded21158>, ([(False, 'band', <xarray.IndexVariable 'band' (band: 3)> array([1, 2, 3])), (False, 'z', <xarray.IndexVariable 'z' (z: 1647861)> array([(193899.75, 4985847.0), (193899.75, 4985391.0), (193899.75, 4984935.0), ..., (805851.75, 4427703.0), (805851.75, 4427247.0), (805851.75, 4426791.0)], dtype=object)), (True, <this-array>, (<function Variable._dask_finalize at 0x7fbadee159d8>, (<function finalize at 0x7fbaf986e8c8>, (), ('band', 'z'), OrderedDict(), None)))], {'z', 'band'}, {'band': 3, 'z': 1647861}, None, None, None), None) kwargs: {} Exception: KeyError(<this-array>,)

distributed.scheduler - ERROR - error from worker inproc://169.154.136.32/2193/2: <this-array>

Do you have any suggestions? I will read up more on stack later to see what else I can learn, but do you have any suggestions? I figure I probably am missing an argument or got something out of order.

Thanks again.