issues

Table actions

1 row where state = "open", type = "issue" and user = 16692099 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

  • issue · 1 ✖

state 1

  • open · 1 ✖

repo 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
694112301 MDU6SXNzdWU2OTQxMTIzMDE= 4406 Threading Lock issue with to_netcdf and Dask arrays bilelomrani1 16692099 open 0     11 2020-09-05T15:49:31Z 2021-12-08T00:06:22Z   NONE      

I have multiple netCDF files that I process and write to a single output file.

```python from pathlib import Path

import xarray as xr

DS = xr.open_mfdataset(Path('L3_data/').glob('*/.nc'), combine='nested', concat_dim='time', decode_times=False, chunks={'time': 200})

DS.to_netcdf('test.nc') ```

I was not able to reproduce the issue with a simple example, here are the files : L3_data.zip. The issue does not appear systematically but sometimes, .to_netcdf blocks indefinitely. When interrupted manually, the trace is the following:

python Traceback (most recent call last): File "test_xarray.py", line 11, in <module> DS.to_netcdf('test.nc') File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/xarray/core/dataset.py", line 1568, in to_netcdf invalid_netcdf=invalid_netcdf, File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/xarray/backends/api.py", line 1090, in to_netcdf writes = writer.sync(compute=compute) File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/xarray/backends/common.py", line 204, in sync regions=self.regions, File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/array/core.py", line 945, in store result.compute(**kwargs) File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/base.py", line 167, in compute (result,) = compute(self, traverse=False, **kwargs) File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/base.py", line 452, in compute results = schedule(dsk, keys, **kwargs) File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/threaded.py", line 84, in get **kwargs File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/local.py", line 475, in get_async key, res_info, failed = queue_get(queue) File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/site-packages/dask/local.py", line 133, in queue_get return q.get() File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/queue.py", line 170, in get self.not_empty.wait() File "/Users/bilelomrani/opt/miniconda3/envs/s5p/lib/python3.7/threading.py", line 296, in wait waiter.acquire() KeyboardInterrupt

Environment:

I'm on Mac OS 10.15.6

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 02:37:09) [Clang 10.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: fr_FR.UTF-8 LOCALE: fr_FR.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.16.0 pandas: 1.1.1 numpy: 1.19.1 scipy: 1.3.1 netCDF4: 1.5.1.2 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.2.1 nc_time_axis: None PseudoNetCDF: None rasterio: 1.0.28 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: None seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.2 conda: None pytest: None IPython: 5.8.0 sphinx: None 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4406/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);