issue_comments

almost no attention is paid to minimizing memory consumption (whether through in-place operations, or more generally minimizing temporary memory usage).

I think we'd be open to fixing this when it doesn't compromise readability. Can you open a new issue with some particularly bad examples?

I wouldn't necessarily say that it's particularly bad, but see the discussion following https://github.com/pydata/xarray/pull/2922#issuecomment-601496897.

almost no attention is paid to minimizing memory consumption (whether through in-place operations, or more generally minimizing temporary memory usage).

I think we'd be open to fixing this when it doesn't compromise readability. Can you open a new issue with some particularly bad examples?

xarray allows using dask arrays for data that don’t fit into memory and dask doesn't support out, https://docs.dask.org/en/latest/array-api.html#dask.array.clip.

I'm not familiar at all with the various numpy interfaces, so I can't offer any input implementation-wise. But as a user, being able to do operations in place (via out or otherwise) is extremely useful when dealing with large arrays under memory constraints. In fact my one "philosophical" beef with xarray is that it seems almost no attention is paid to minimizing memory consumption (whether through in-place operations, or more generally minimizing temporary memory usage).

__array_wrap__ doesn't allow specifying any kwargs (as far as I can tell), it has two parameters: obj and context. I'd have to read up on this, but obj seems to be the data (from asarray or __array_prepare__?) and context seems to be None most of the time.

In any case, I think we should try to replace __array_wrap__ with __array_function__ (or __array_module__ / __array_namespace__?) soon.

Also, I agree on being consistent regarding the out parameter.

Thanks @keewis , that makes sense.

And ensuring I return to @seth-p 's original issue — I would be up for adding out if we're violating the __array_wrap__ standard by not having all its kwargs.

Otherwise I would vote weakly against out throughout xarray's API, and strongly against it out on an arbitrary subset of methods.

Thanks for the example @raybellwaves

Supporting np.{func} would be good — I have only partially followed the interop with numpy — e.g. __array_ufunc__ (edit: __array_function__) — is anyone familiar with whether we're violating the standards by not having an out arg with clip?

FYI, we don't generally have out args, but we do have them in nanops.py:

xarray/core/nanops.py: 78: def nanmin(a, axis=None, out=None): 86: def nanmax(a, axis=None, out=None): 112: def nansum(a, axis=None, dtype=None, out=None, min_count=None): 137: def nanmean(a, axis=None, dtype=None, out=None): 151: def nanmedian(a, axis=None, out=None): 169: def nanvar(a, axis=None, dtype=None, out=None, ddof=0): 178: def nanstd(a, axis=None, dtype=None, out=None, ddof=0): 184: def nanprod(a, axis=None, dtype=None, out=None, min_count=None): 193: def nancumsum(a, axis=None, dtype=None, out=None): 199: def nancumprod(a, axis=None, dtype=None, out=None):

clip dispatches through __array_function__ (it's not a real "ufunc"), so we don't really support that (there's a issue for adding __array_function__, but no progress yet). In general I think we should only support functions which don't have a axis parameter and return NotImplemented for all others, which means clip would be a good candidate.

We currently do have some support for numpy.clip(arr, ...) through __array_wrap__ (so no support for Dataset), not sure if that even allows deviating from a standard?

FWIW. This showed up in xskillscore as we were doing np.clip(xarray object, min, min). We updated the code to do xarray object.clip(min, max) which we probably should have been doing in the first place (https://github.com/xarray-contrib/xskillscore/pull/309#issue-634256273)

What is the case for having out kwargs?

It lets you reuse memory you already have. In particular for a simple operation like clip, you can do it in-place: da.clip(..., out=da.values). Very useful if you deal with lots of data and memory is a concern.

FYI I'm out on paternity leave for a few weeks.

Generally IIRC out hasn't been part of the public API, and only present because of our delegating to numpy for some functions.

What is the case for having out kwargs?