issue_comments

@dcherian Just to be complete, I thought the following one-liner would work as well:

sums, x = np.histogram(latitude, bins, weights=array)

but apparently it produces slightly different results for reasons I don't understand

the IDL histogram function but in numpy.

Apparently not as awesome!

Yeah, the present solution is less general, but most of the time I'm just counting stuff, and this is much faster than what I was doing, so I'm happy ;-)

I'm getting a bit off topic now, but ...

Apparently not as awesome!

@dcherian Thanks for bringing back fond memories of the past. I still have @davidwfanning's IDL books on the shelf. And for sure it was a great pleasure reading @jdtsmith's IDL tricks and trying to understand those helped a lot. Great stuff.

the IDL histogram function but in numpy.

Apparently not as awesome!

@dcherian I just tested your numpy suggestions, and I'm getting 100x speedups compared to my naive numpy approach (~200µs vs ~20ms). Thankyouthankyouthankyou! I've been doing this for years, I can't believe I've never run into that particular solution. It's like the IDL histogram function but in numpy. I'm going to use this like crazy Thanks again

Yes that's right.

For this simple problem you could combine np.digitize and np.bincount to do it much quicker.

python group_idx = np.digitize(latitude, bins) sums = np.bincount(group_idx, weights=array)

And then wrap this using apply_ufunc. See https://github.com/ml31415/numpy-groupies/blob/412be938dcdfd74c6d673dd29012d18dc25dc94f/numpy_groupies/aggregate_numpy.py#L8-L28 for inspiration.

@dcherian this means that xarray's groupby_bins will always be slow unless flox is installed, correct? I have unfortunately little or no say on what packages are installed on the system that runs my code.

On xarray main with flox installed:

``` python import numpy as np import xarray as xr display(xr.version)

N = 3728 ds = xr.Dataset() ds["latitude"] = ("x", 0 + 20 * np.random.standard_normal(N)) ds["data"] = ("x", 0 + 100 * np.random.standard_normal(N))

%timeit ds.groupby_bins("latitude", np.arange(-40, 40, 0.1)).sum() ``` 50.3 ms ± 203 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

You could try it on our pre-release (https://docs.xarray.dev/en/latest/whats-new.html#v2022-06-0rc0-9-june-2022) or use xhistogram which should be faster even.