issue_comments

@shoyer, are you happy for me to go ahead and change unstack() to respect the order of the first found series?

Thanks, didn't know

https://gist.github.com/crusaderky/002ba64ee270164931d32ea3366dce1f

Fixed in attachment. The code uses the first found series as the order.

proper_unstack.zip

I see. I'll see if I can think a good way to cope with your two examples. BTW, my code above is buggy as it blindly assumes that the first dim is also the outermost.

This preamble should be integrated inside unstack():

``` python import operator from functools import reduce

def proper_unstack(array, dim):

# Regenerate Pandas multi-index to be ordered by appearance
# TODO: check that the stacked coords repeat periodically
# TODO: write a faster/cleaner algorithm using numpy
mindex = array.coords[dim].to_pandas().index

levels = []
labels = []
for dim_i, (levels_i, labels_i) in enumerate(zip(mindex.levels, mindex.labels)):
    step_inner = reduce(operator.mul, (len(lvl) for lvl in mindex.levels[dim_i + 1:]), 1)
    step_outer = reduce(operator.mul, (len(lvl) for lvl in mindex.levels[:dim_i]), 1)

    levels.append([levels_i[labels_i[j]] for j in range(0, levels_i.size * step_inner, step_inner)])
    labels.append(reduce(operator.add, ([j] * step_inner for j in range(levels_i.size))) * step_outer)

mindex = pandas.MultiIndex(levels, labels, names=mindex.names)
array = array.copy()
array.coords[dim] = mindex
return array.unstack(dim)

proper_unstack(a, 'dim_0') ```

<xarray.DataArray (x: 2, count: 4)> array([[0, 1, 2, 3], [4, 5, 6, 7]]) Coordinates: * x (x) object 'x1' 'x0' * count (count) object 'first' 'second' 'third' 'fourth'

the order of appearance should be what dictates the output.

in the worst case (e.g., random order for the MultiIndex) we'll have this issue no matter what rule we pick for assigning unstacked coordinates.

Not true. Using the order of appearance requires you to do a pick-by-index on the index. At the moment, you're doing a pick-by-index on the data.

this workaround works:

python index2 = pandas.MultiIndex( levels=[['x0', 'x1'], ['first', 'second', 'third', 'fourth']], labels=[[0,0,0,0,1,1,1,1], [0,1,2,3,0,1,2,3]], names=['x', 'count']) xarray.DataArray(pandas.Series(list(range(8)), index2)).unstack('dim_0')

<xarray.DataArray (x: 2, count: 4)> array([[0, 1, 2, 3], [4, 5, 6, 7]], dtype=int64) Coordinates: * x (x) object 'x0' 'x1' * count (count) object 'first' 'second' 'third' 'fourth'

However, I think that the whole thing is incredibly convoluted. Namely, because everything looks good both if you visualize the original pandas Series/DataFrame, as well as the stacked DataArray. unstack() is causing an internal technicality of pandas to produce real change in the data.

I came through this issue because I am using pandas to load a multi-index CSV from disk, and then convert it to a n-dimensional xarray. In this situation, I have no control over the multiindex - short of manually rebuilding it after the CSV load. The pandas dataframe looks right, the stacked xarray looks right, the unstacked xarray gets magically sorted :$

Also I don't understand why you say there's no performance implications. You're basically doing a pick-by-index rebuild of the array, which does potentially random access to the whole input array - thus nullifying the benefits of the CPU cache. This is compared to a numpy.ndarray.reshape(), which has the cost of a memcpy().

I was going to add something about doing pick-by-index with a dask array will be even worse, when I realised that multiindex does not work at all when you chunk()... :(