issues

Table actions

1 row where state = "open", type = "issue" and user = 31093031 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

  • issue · 1 ✖

state 1

  • open · 1 ✖

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
460254571 MDU6SXNzdWU0NjAyNTQ1NzE= 3041 nc file locked by xarray after (double) da.compute() call delandmeterp 31093031 open 0     5 2019-06-25T07:19:35Z 2021-07-21T14:29:57Z   NONE      

Hi, I've recently started to use dask.array to compute and load only chunks of a huge dataset for the development of Parcels. I've run into an issue of locked netcdf files while running multiple tests using py.test, and I think my bug should be related to the reduced issue that I reproduce hereinafter:

Problem Description

``` import numpy as np import xarray as xr import dask.array as da

def create_ds(): temp = 15 + 8 * np.random.randn(2, 2) precip = 10 * np.random.rand(2, 2) lon = [[-99.83, -99.32], [-99.79, -99.23]] lat = [[42.25, 42.21], [42.63, 42.59]] ds = xr.Dataset({'temperature': (['x', 'y'], temp), 'precipitation': (['x', 'y'], precip)}, coords={'lon': (['x', 'y'], lon), 'lat': (['x', 'y'], lat)}) ds.to_netcdf('test.nc')

def dask_op(): ds = xr.open_dataset('test.nc') temp = da.from_array(ds.temperature, chunks='auto') temp.compute() temp.compute()

create_ds() dask_op() dset = xr.Dataset() dset.to_netcdf('test.nc') ```

Output

The last line of the mini-code crashes since test.nc is still locked. Of course the problem can be circumvented by closing the dataset after last call to temp.compute, but this operation is not necessary if temp.compute() is called only once?

Working and not working alternatives

If I replace the 2 temp.compute() lines by either: temp.compute() or temp.compute() temp.compute() ds.close The code passes, but with those blocks: temp.compute() temp.compute() or ds.close temp.compute() temp.compute() ds.close it doesn't.

This issue was also posted on dask, where I was advised to also ask for help here. Any hint about the reason for this issue? Thanks!

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3041/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);