Speaking a bit to things like cumprod, it's hard to support those natively with sparse data structures in many cases (at least as things stand in the current Numba framework).

While that doesn't apply in the case of cumprod, PyData/Sparse also has a policy that if the best algorithm available is a dense one, we simply raise an error, and the user should densify explicitly to avoid filling all available RAM or getting obscure MemoryErrors.

For ffill specifically, you would get a dense array out anyway, so there's no point to keeping it sparse, unless one did something like run-length-encoding or similar.

As for the size issue, PyData/Sparse provides the nbytes attribute which could be helpful in determining size.

I would very much prefer not to be recorded.

Thank you guys for the amazing investigation into the issues here. 🤗

I was basically pinpointed at the issues from my point of view.

I have fixed the reported issues and released version 0.13.0 with the fixes included.

Third and final issue is when numpy.broadcast_to is applied to the output of zeros_like:

```

import sparse

s = sparse.COO.from_numpy([0, 0, 1, 2])

np.broadcast_to(np.zeros_like(s.todense(), shape=()), (3, ))

array([0, 0, 0])

np.broadcast_to(np.zeros_like(s, shape=()), (3, ))

ValueError: The data length does not match the coordinates given.

len(data) = 0, but 3 coords specified.

```

This seems like a genuine bug, please file it on our tracker if possible.

I'd also be happy to attend. Keep in mind I'm in the CET timezone.

Hi. Yes, it’d be nice if we had a meta issue I could then open separate issues for for sllearn implementations.

Performance is not ideal, and I realise that. However I’m working on a more generic solution to performance as I type.

@rabernat I can attend remotely.

uarray/unumpy is shaping up nicely. 😄

Let's move this discussion over to hameerabbasi/arrayish#1. But, in summary, I got the impression that the community in general is unhappy with the name "duck arrays".

I've written it up and already released version 0.0.1 on PyPI, except concatenate and stack (which need TypedSequence). I can still change the name, but I'd rather not.

Also, import duckarray as da conflicts with import dask.array as da.

I've created one, as per your e-mail: https://github.com/hameerabbasi/arrayish

The name is inspired from a recent discussion about this on the Numpy mailing list.

By minimal library, I'm assuming you mean something of the sort discussed about abstract arrays? What functionality would such a library have?

Things look fine now. :-)

xref pydata/sparse#103

Maybe submit a PR? We could all use this. Does it support variable-length arguments?

I have experience with Nginx, which PyData uses (according to some HTTP headers).

Which really is totally fine -- this is all a stop gap measure until NumPy itself supports this sort of duck typing.

You're assuming here most users of XArray would be using a recent version of Numpy... Which is a totally fine assumption IMO. We make the same one for sparse.

However, consider that some people may be using something like conda, which (because of complex dependencies and all) may end up delaying updates (both for Numpy and XArray).

I guess however; if people really wanted the updates they could use pip.

so I'm not sure it's worth enshrining in multipledispatch either

I would say a little clean-up with some extra decorators for exactly this purpose may be in order, that way, individual wrapping functions aren't needed.

Something like @starargswrapper that would just cast to list, and call the VarArgs version.

Actually it'd be nice to have something like @dispatch(int, str, StarArgs[int]).

@llllllllll How hard would it be to make this work for star-args? I realize you could just add an extra wrapper but it'd be nice if you didn't have to.

Is there a way to handle kwargs (not with types, but ignoring them)?

This might even help us out in Sparse for dispatch with scipy.sparse.spmatrix, numpy.ndarray, etc.

Another benefit to this would be that if XArray didn't want to support a particular library in its own code, the library itself could add the hooks.

How about something like checking inside a list if something is top priority, then call a, if second priority, call b, etc.

Can't some wild metaprogramming make it so that [1.0, 'foo'] itself is an instance of VarArgs[float, str] (or be converted?)

Then I would suggest something like the following for hooks (omitting imports):

```python

Registered in order of priority

xarray.interfaces.register('DaskArray', lambda ar: isinstance(ar, da.array)) xarray.hooks.register('nansum', 'DaskArray', da.nansum)

xarray.interfaces.register('SparseArray', lambda ar: isinstance(ar, sparse.SparseArray)) xarray.hooks.register('nansum', 'SparseArray', sparse.nansum) ```

And then, in code, call the appropriate nansum instead of np.nansum:

python nansum = xarray.hooks.get(arr, 'nansum')

If you need help, I'd be willing to give it. :-) But I'm not a user of XArray, so I don't really understand the use-cases or codebase.