issue_comments

I'd very happy to start working on #1603, but unfortunately I wish I could have more time for this at the moment (and before this, it is also probably a good idea that I continue the work in #1820, which has stalled for quite a while).

Thanks @shoyer, I find this feature extremely useful as I keep running into use cases where I can use it. Thanks for the update, given changes to xarray it sounds like the prudent course of action is as you outline. Thanks again for the quick reply!

@pwolfram Personally, I am waiting to implement this until after https://github.com/pydata/xarray/issues/1603. I'm pretty sure that the alternative model for indexes proposed there will make this far easier to implement.

@shoyer, it looks like your list above is the place to start from your branch, correct?

Just to refresh here-- what needs done to finish this off?

@chunweiyuan - yes, you are welcome to give this a shot.

This functionality is quite important to us as well. How might we help? Should I simply fork groupby-multiple-args?

@pwolfram Unfortunately nothing from my side yet.

I don't have any progress to report since my last comment.

@RafalSkolasinski and @shoyer, can I please get an update on this PR? This is something we need sometime soon too (cc @milenaveneziani).

@RafalSkolasinski Sure, here is the current list:

1. The implementation should probably switch to use set_index() (after stacking) to combine groupby arguments into a single MultiIndex, instead of the current version which constructs the MultiIndex explicitly in GroupBy.__init__, duplicating a lot of the set_index code.
2. We need to consider how to handle multiple group-by arguments along orthogonal dimensions. For example, consider the 4x4 array from the gist at the top of this PR where grouping is done over 2x2 blocks. With the current PR, the array is flattened, so iteration is done over flat arrays of length 4, but we probably want to actually iterate over 2x2 arrays that match the shape of the original data. This could be done either by stacking/unstacking at each iteration step (slow, but would work) or by making groupby truly do indexing and concatenation over multiple dimensions at once (certainly faster and more elegant, but significantly more work to implement).
3. The change also needs to be rebased to account for changes on master, but by the time you account for the above changes there may not be much of the current version remaining.
4. We also need comprehensive test coverage for the desired behavior.

@shoyer I am considering contributing to this feature. Could you give me more details what needs to be done?

@RafalSkolasinski This pull request was mostly working, but still needs some significant work to clean it up and update it to the current version of the codebase.

I don't think anyone is working on it currently (I'm not) but I'm sure someone will get to it eventually.

Hi, is there any active work on that feature? It would be really cool to have it.

@rabernat I updated the top post with examples. So yes, for your example, the coordinates of the output would have every unique combination of x and y. More generally, something like ds.groupby(['x', 'y']).mean() will be equivalent to ds.mean('z') for a dataset with dimensions (x, y, z). I think this is the only sane way to define these grouped operations.

Once we figure out squeezing out grouped/stacked dimensions (not quite working yet), this will let us write things like ds.groupby(['x', 'y']).apply(calculate_trend) or better yet with group_over, ds.group_over('time').apply(calculate_trend).

This looks like a really useful addition. It would be useful for me to have an example of how this is supposed to work. The tests are a starting point, but perhaps kind of trivial cases.

If I create the following 2D dataset:

python ds = xr.Dataset({'foo': (['x','y'], np.random.rand(2,4))})

and then do

python ds.groupby(['x','y']).sum()

What should the coordinates of the output be? Every unique combination of x and y?