html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/4085#issuecomment-631670146,https://api.github.com/repos/pydata/xarray/issues/4085,631670146,MDEyOklzc3VlQ29tbWVudDYzMTY3MDE0Ng==,2599958,2020-05-20T19:11:56Z,2020-05-20T19:11:56Z,NONE,"The problem was only with very large arrays, so difficult to reproduce here.

Fortunately, when I just now updated to the very latest xarray/dask, the problem seems to have gone away.

I was puzzled because things worked fine -- as expected -- but only for smaller arrays. I will close this issue.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474
https://github.com/pydata/xarray/issues/4085#issuecomment-631658737,https://api.github.com/repos/pydata/xarray/issues/4085,631658737,MDEyOklzc3VlQ29tbWVudDYzMTY1ODczNw==,2448579,2020-05-20T18:50:50Z,2020-05-20T18:50:50Z,MEMBER,Something is very wrong here. is this reproducible?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474
https://github.com/pydata/xarray/issues/4085#issuecomment-631636660,https://api.github.com/repos/pydata/xarray/issues/4085,631636660,MDEyOklzc3VlQ29tbWVudDYzMTYzNjY2MA==,34353851,2020-05-20T18:07:22Z,2020-05-20T18:07:22Z,NONE,"I use  http://xarray.pydata.org/en/stable/generated/xarray.apply_ufunc.html
because
it is faster.

El mié., 20 may. 2020 a las 20:01, Javier Ruano (<javier.ruanno@gmail.com>)
escribió:

> if you append compute() it should not be a lazy operation.
> But my advice is like user only.
>
> El mié., 20 may. 2020 a las 19:51, Rob Hetland (<notifications@github.com>)
> escribió:
>
>> I have a large DataSet, including these DataArrays:
>>
>> <xarray.DataArray 'temp' (ocean_time: 1325, s_rho: 30, eta_rho: 602, xi_rho: 677)>
>> dask.array<concatenate, shape=(1325, 30, 602, 677), dtype=float32, chunksize=(1, 1, 602, 677)>
>>
>> and
>>
>> <xarray.DataArray 'zeta' (ocean_time: 1325, eta_rho: 602, xi_rho: 677)>
>> dask.array<concatenate, shape=(1325, 602, 677), dtype=float32, chunksize=(1, 602, 677)>
>>
>> (The coordinates and attributes excluded for brevity, but they match in
>> the right ways.)
>>
>> When I do math operations with the 4D DataArray (temp) and 3D DataArray
>> (zeta), no problem:
>>
>> ds.zeta * ds.temp
>>
>> <xarray.DataArray (ocean_time: 1325, eta_rho: 602, xi_rho: 677, s_rho: 30)>
>> dask.array<mul, shape=(1325, 602, 677, 30), dtype=float32, chunksize=(1, 602, 677, 1)>
>>
>> This returns an object instantly, and the result is lazily evaluated.
>> However, if I just try to add temp to itself,
>>
>> ds.temp + ds.temp
>>
>> this fails (eventually) as my medium sized computer runs out of memory,
>> since it starts to evaluate the numbers as if I did a compute() or asked
>> for the values. Note 2*ds.temp or ds.temp**2 is lazily evaluated, and
>> returns an object instantly. Chunk size does not seem to be an issue, as I
>> have tried a number of reasonable choices without success.
>>
>> Why can't such simple math operations between two large arrays also be
>> lazily evaluated?
>>
>> —
>> You are receiving this because you are subscribed to this thread.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/pydata/xarray/issues/4085>, or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/AIGDFO2ZUJJOVNCN4JRJYDTRSQKBTANCNFSM4NGFVQRQ>
>> .
>>
>
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474
https://github.com/pydata/xarray/issues/4085#issuecomment-631633988,https://api.github.com/repos/pydata/xarray/issues/4085,631633988,MDEyOklzc3VlQ29tbWVudDYzMTYzMzk4OA==,34353851,2020-05-20T18:01:42Z,2020-05-20T18:01:42Z,NONE,"if you append compute() it should not be a lazy operation.
But my advice is like user only.

El mié., 20 may. 2020 a las 19:51, Rob Hetland (<notifications@github.com>)
escribió:

> I have a large DataSet, including these DataArrays:
>
> <xarray.DataArray 'temp' (ocean_time: 1325, s_rho: 30, eta_rho: 602, xi_rho: 677)>
> dask.array<concatenate, shape=(1325, 30, 602, 677), dtype=float32, chunksize=(1, 1, 602, 677)>
>
> and
>
> <xarray.DataArray 'zeta' (ocean_time: 1325, eta_rho: 602, xi_rho: 677)>
> dask.array<concatenate, shape=(1325, 602, 677), dtype=float32, chunksize=(1, 602, 677)>
>
> (The coordinates and attributes excluded for brevity, but they match in
> the right ways.)
>
> When I do math operations with the 4D DataArray (temp) and 3D DataArray
> (zeta), no problem:
>
> ds.zeta * ds.temp
>
> <xarray.DataArray (ocean_time: 1325, eta_rho: 602, xi_rho: 677, s_rho: 30)>
> dask.array<mul, shape=(1325, 602, 677, 30), dtype=float32, chunksize=(1, 602, 677, 1)>
>
> This returns an object instantly, and the result is lazily evaluated.
> However, if I just try to add temp to itself,
>
> ds.temp + ds.temp
>
> this fails (eventually) as my medium sized computer runs out of memory,
> since it starts to evaluate the numbers as if I did a compute() or asked
> for the values. Note 2*ds.temp or ds.temp**2 is lazily evaluated, and
> returns an object instantly. Chunk size does not seem to be an issue, as I
> have tried a number of reasonable choices without success.
>
> Why can't such simple math operations between two large arrays also be
> lazily evaluated?
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pydata/xarray/issues/4085>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AIGDFO2ZUJJOVNCN4JRJYDTRSQKBTANCNFSM4NGFVQRQ>
> .
>
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,621968474