issues: 621968474

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
621968474 MDU6SXNzdWU2MjE5Njg0NzQ= 4085 lazy evaluation of large arrays fails 2599958 closed 0     4 2020-05-20T17:51:02Z 2020-05-20T19:11:57Z 2020-05-20T19:11:56Z NONE      

I have a large DataSet, including these DataArrays:

<xarray.DataArray 'temp' (ocean_time: 1325, s_rho: 30, eta_rho: 602, xi_rho: 677)>
dask.array<concatenate, shape=(1325, 30, 602, 677), dtype=float32, chunksize=(1, 1, 602, 677)>

and

<xarray.DataArray 'zeta' (ocean_time: 1325, eta_rho: 602, xi_rho: 677)>
dask.array<concatenate, shape=(1325, 602, 677), dtype=float32, chunksize=(1, 602, 677)>

(The coordinates and attributes excluded for brevity, but they match in the right ways.)

When I do math operations with the 4D DataArray (temp) and 3D DataArray (zeta), no problem:

ds.zeta * ds.temp

<xarray.DataArray (ocean_time: 1325, eta_rho: 602, xi_rho: 677, s_rho: 30)>
dask.array<mul, shape=(1325, 602, 677, 30), dtype=float32, chunksize=(1, 602, 677, 1)>

This returns an object instantly, and the result is lazily evaluated. However, if I just try to add temp to itself,

ds.temp + ds.temp

this fails (eventually) as my medium sized computer runs out of memory, since it starts to evaluate the numbers as if I did a compute() or asked for the values. Note 2*ds.temp or ds.temp**2 is lazily evaluated, and returns an object instantly. Chunk size does not seem to be an issue, as I have tried a number of reasonable choices without success.

Why can't such simple math operations between two large arrays also be lazily evaluated?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4085/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables