Following @willirath 's suggestion in #4804, I've changed https://github.com/pydata/xarray/blob/master/xarray/core/computation.py#L1373_L1375 so that Dask doesn't hold all chunks in memory

• [x] Closes (more of) #4804, specifically this comment
• [x] Passes pre-commit run --all-files

ignore this, I did the branching wrong...

Was reading the discussion here and thought I'd draft a PR to implement some of the changes people suggested. It seems like the main problem is that currently in computation.py, if not valid_values.all() is not a lazy operation, and so can be a bottleneck for very large dataarrays. To get around this, I've lifted some neat tricks from #4559 so that valid_values remains a dask array.

• [x] Closes #4804
• [x] Tests added
• [x] Passes pre-commit run --all-files
• [x] User visible changes (including notable bug fixes) are documented in whats-new.rst

What happened: When interpolating a multidimensional array, I used the fill_value=None kwarg, as in the scipy docs it says that If None, values outside the domain are extrapolated.

What you expected to happen: However, this did not work and the array was filled with nans instead of being extrapolated. I had to explicitly add fill_value='extrapolate' in order to get extrapolation to work.

However, as far as I can tell, in this context interp_like() should be calling scipy.interpolate.interpn(), which doesn't seem to require fill_value='extrapolate'?

I might be misunderstanding something here, but if not then I think there might be a bug? Happy to be corrected if I've misunderstood the docs!

Minimal Complete Verifiable Example:

```python

Put your MCVE code here

import xarray as xr import numpy as np

da = xr.DataArray( np.array(([1,2,3,4,0],[1,2,3,9,0])), dims=["x", "y"], coords={"x": [0,1],"y": [0,1,2,3,4]} )

da2 = xr.DataArray( np.array(([2,3,0,4,5,6],[2,3,0,4,5,6],[2,3,0,4,5,6])), dims=["x", "y"], coords={"x": [0,1,2],"y": [0,1,2,3,4,5]} )

print(da.interp_like(da2)) <xarray.DataArray (x: 3, y: 6)> array([[ 1., 2., 3., 4., 0., nan], [ 1., 2., 3., 9., 0., nan], [nan, nan, nan, nan, nan, nan]]) Coordinates: * x (x) int64 0 1 2 * y (y) int64 0 1 2 3 4 5

print(da.interp_like(da2, kwargs={'fill_value':None})) <xarray.DataArray (x: 3, y: 6)> array([[ 1., 2., 3., 4., 0., nan], [ 1., 2., 3., 9., 0., nan], [nan, nan, nan, nan, nan, nan]]) Coordinates: * x (x) int64 0 1 2 * y (y) int64 0 1 2 3 4 5

print(da.interp_like(da2, kwargs={'fill_value':'extrapolate'})) <xarray.DataArray (x: 3, y: 6)> array([[ 1., 2., 3., 4., 0., -4.], [ 1., 2., 3., 9., 0., -9.], [ 1., 2., 3., 14., 0., -14.]]) Coordinates: * x (x) int64 0 1 2 * y (y) int64 0 1 2 3 4 5 ```

Environment:

Fixing typo in recent PR #4089

• [x] Closes https://github.com/pydata/xarray/pull/4089#issuecomment-634157768
• [x] Passes isort -rc . && black . && mypy . && flake8

PR for the xr.cov() and xr.corr() functionality which others have been working on. Most code adapted from @r-beer in PR #3550.

TODO: - [x] ~Write a reasonable set of tests, maybe not using pandas as a benchmark? (See https://github.com/pydata/xarray/issues/3784#issuecomment-633145887) Will probably need some help with this~

CHECKLIST: - [x] Closes #3784, #3550, #2652, #1115 - [x] Tests added - [x] Passes isort -rc . && black . && mypy . && flake8 ~(something wrong with docs though??)~ - [x] Fully documented, including whats-new.rst for all changes and api.rst for new API

Adding chunks='auto' option to Dataset.chunk().

• [x] Closes #4055
• [x] Tests added in test_dask.py
• [x] Passes isort -rc . && black . && mypy . && flake8
• [x] Updated whats-new.rst for changes

Hi there,

Hopefully this turns out to be a basic issue, but I was wondering why the chunks='auto' that dask seems to provide (https://docs.dask.org/en/latest/array-chunks.html#automatic-chunking) isn't an option for xarray? I'm not 100% sure of how dask decides how to automatically chunk its arrays, so maybe there's a compatibility issue?

I get the impression that the dask method automatically tries to prevent the issues of "too many chunks" or "too few chunks" which can sometimes happen when choosing chunk sizes automatically. If so, it would maybe be a useful thing to include in future versions?

Happy to be corrected if I've misunderstood something here though, still getting my head around how the dask/xarray compatibility really works...

Cheers!

I came across this StackOverflow thread about applying linear regression across multi-dimensional arrays and there were some very efficient, xarray-based solutions suggested to the OP which leveraged vectorized operations. I was wondering if there was any interest in adding a function like this to future releases of xarray? It seems like a very neat and useful functionality to have, and not just for meteorological applications!

https://stackoverflow.com/questions/52108417/how-to-apply-linear-regression-to-every-pixel-in-a-large-multi-dimensional-array

Most answers draw from this blog post: https://hrishichandanpurkar.blogspot.com/2017/09/vectorized-functions-for-correlation.html