issues: 460254571

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
460254571 MDU6SXNzdWU0NjAyNTQ1NzE= 3041 nc file locked by xarray after (double) da.compute() call 31093031 open 0     5 2019-06-25T07:19:35Z 2021-07-21T14:29:57Z   NONE      

Hi, I've recently started to use dask.array to compute and load only chunks of a huge dataset for the development of Parcels. I've run into an issue of locked netcdf files while running multiple tests using py.test, and I think my bug should be related to the reduced issue that I reproduce hereinafter:

Problem Description

``` import numpy as np import xarray as xr import dask.array as da

def create_ds(): temp = 15 + 8 * np.random.randn(2, 2) precip = 10 * np.random.rand(2, 2) lon = [[-99.83, -99.32], [-99.79, -99.23]] lat = [[42.25, 42.21], [42.63, 42.59]] ds = xr.Dataset({'temperature': (['x', 'y'], temp), 'precipitation': (['x', 'y'], precip)}, coords={'lon': (['x', 'y'], lon), 'lat': (['x', 'y'], lat)}) ds.to_netcdf('test.nc')

def dask_op(): ds = xr.open_dataset('test.nc') temp = da.from_array(ds.temperature, chunks='auto') temp.compute() temp.compute()

create_ds() dask_op() dset = xr.Dataset() dset.to_netcdf('test.nc') ```

Output

The last line of the mini-code crashes since test.nc is still locked. Of course the problem can be circumvented by closing the dataset after last call to temp.compute, but this operation is not necessary if temp.compute() is called only once?

Working and not working alternatives

If I replace the 2 temp.compute() lines by either: temp.compute() or temp.compute() temp.compute() ds.close The code passes, but with those blocks: temp.compute() temp.compute() or ds.close temp.compute() temp.compute() ds.close it doesn't.

This issue was also posted on dask, where I was advised to also ask for help here. Any hint about the reason for this issue? Thanks!

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3041/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     13221727 issue

Links from other tables