issue_comments

If you haven't already, would be good if those running into issues here could look over the Array API. This is still something that is being worked on, but the goal is to standardize Array APIs. If there are things missing from that, it would be good to hear about them in a new issue.

There are also some relevant and very interesting PyTorch development discussions; there are a lot of Tensor subclasses in the making and thought being put into how those interact: - https://dev-discuss.pytorch.org/t/state-of-pytorch-core-september-2021-edition/332#alternative-tensors-5 - https://pytorch-dev-podcast.simplecast.com/episodes/tensor-subclasses-and-liskov-substitution-principle - https://dev-discuss.pytorch.org/t/functorch-levels-as-dynamically-allocated-classes/294

I'd like to attend on behalf of QuTiP. I'll likely mostly listen -- QuTiP is not directly affected in the way that xarray, Dask, cupy, etc are, but we're users of potentially all of these array types (and already explicitly support numpy, CuPy, and our own sparse array format) and we are facing similar issues of our own (i.e. users of QuTiP are asking us to develop a __qutip_qobj__ style protocol similar to numpy's and we would like to learn the lessons of the last decade of numpy rather than repeat the steps ourselves in the coming one).

Will try to join.

I'd also like to attend, primarily just to learn.

Maybe too soon to ask, but do we have a link to the video call?

👂🏾

* scipp (@SimonHeybrock, xref [NEP 18, physical units, uncertainties, and the scipp library? #3509](https://github.com/pydata/xarray/issues/3509))

Thanks! I am definitely interested.

Interesting - could you say a bit more? Looking at these two issues, it seemed more like the question was simply on hold until someone who wanted it badly enough came along?

There is a significant backwards compatibility break when a library adds __array_ufunc__ and __array_function__. JAX maintainers were not comfortable with that. @shoyer wrote https://numpy.org/neps/nep-0037-array-module.html as a follow-up largely because of that. That NEP is effectively superceded by the array API standard (https://data-apis.org/array-api/latest/ and NEP 47). PyTorch has decided to adopt that and implementation is well underway. Experimental support in NumPy will land next week (complete except for linalg). CuPy and MXNet plan to follow that NumPy implementation. JAX and Dask not yet decided I believe, but likely to follow NumPy as well.

Canonical/minimal API of a "duck array" and how to detect it (though may be superseded by NEPs 30 and 47 among others)

This is basically what the array API standard provides (most functions follow NumPy, with deviations mostly where other libraries were already deviating because they could implement something on GPU, with a JIT, or with a non-strided memory model). __array_function__ has worked quite well for CuPy and Dask, because they follow the NumPy API almost to the letter, with only a couple of exceptions (e.g. 0-D array instead of array scalars in CuPy). JAX, PyTorch and MXNet all deviate much more, and since the NumPy API is not very well-defined (there's 1500+ public objects plus more semi-public ones), you'd have no guarantees about what works and what doesn't.

That said, __array_ufunc__ and __array_function__ aren't going anywhere. The RAPIDS ecosystem is invested in it and I believe largely happy with it. So adding Xarray and Pint to the mix sounds potentially interesting.

• Awkward? (@jpivarski)

I'm interested. Let us know when the time will be or if there's a poll for picking a time. Thanks!

Happy to hop on a call for this as well, thanks for organizing all of this @TomNicholas!

I'm happy to join, seems interesting. And yes, I can say something about PyTorch. There probably isn't much to say though - PyTorch is unlikely to adopt __array_function__ at this point, just like JAX. And it doesn't seem critical for this hierarchy anyway - the fundamental array objects (PyTorch/CuPy/NumPy/Sparse/JAX arrays or tensors) do not have or need a class hierarchy, they are all at the bottom and should not be mixed.

The key thing here seems to be Dask <-> Xarray <-> Pint, unless I'm missing something?