issue_comments

Table actions

6 rows where issue = 1385031286 and user = 3487237 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 1

  • kthyng · 6 ✖

issue 1

  • open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 · 6 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1494957700 https://github.com/pydata/xarray/issues/7079#issuecomment-1494957700 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85ZGz6E kthyng 3487237 2023-04-03T20:45:42Z 2023-04-03T20:45:42Z NONE

I'm not really sure what to think any more — we have had a real, consistent issue that seemed to fit the description of this issue which went away with one of the fixes above (using single threading), but using local files at the moment seems to remove the error even with the current version of xarray and either parallel option.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286
1494950846 https://github.com/pydata/xarray/issues/7079#issuecomment-1494950846 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85ZGyO- kthyng 3487237 2023-04-03T20:39:02Z 2023-04-03T20:39:02Z NONE

Ok I downloaded the two files and indeed there is no error with parallel=True nor parallel=False.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286
1494542372 https://github.com/pydata/xarray/issues/7079#issuecomment-1494542372 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85ZFOgk kthyng 3487237 2023-04-03T15:31:54Z 2023-04-03T15:31:54Z NONE

@jhamman Yes, using the PR version of xarray, with parallel=True I met the error but with parallel=False I did not.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286
1492670307 https://github.com/pydata/xarray/issues/7079#issuecomment-1492670307 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85Y-Fdj kthyng 3487237 2023-03-31T22:14:06Z 2023-03-31T22:14:06Z NONE

I was able to reproduce the error with the current version of xarray and then have it work with the new version. Here is what I did:

Make new environment conda create -n test_xarray xarray netcdf4 dask

Check version ``` (test_xarray) kthyng@adams ~ % conda list xarray

packages in environment at /Users/kthyng/miniconda3/envs/test_xarray:

Name Version Build Channel

xarray 2023.3.0 pyhd8ed1ab_0 conda-forge ```

In python: import xarray as xr urls = ["https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/WCOFS/MODELS/2023/03/31/nos.wcofs.2ds.n001.20230331.t03z.nc", "https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/WCOFS/MODELS/2023/03/31/nos.wcofs.2ds.n002.20230331.t03z.nc"] xr.open_mfdataset(urls) returns the following the first time xr.open_mfdataset(urls) is run but the second time it runs fine. OSError: [Errno -70] NetCDF: DAP server error: 'https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/WCOFS/MODELS/2023/03/31/nos.wcofs.2ds.n002.20230331.t03z.nc'

Next I used the PR version of xarray and reran the code above and then it was able to read in ok on the first try.

Note: after a week or so those files won't work and will have to be updated with something more current but the pattern to use is clear from the file names.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286
1490420229 https://github.com/pydata/xarray/issues/7079#issuecomment-1490420229 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85Y1gIF kthyng 3487237 2023-03-30T14:36:39Z 2023-03-30T14:36:39Z NONE

@jhamman Sorry for my delay — I started this the other day and got waylaid. I'll try to get back to it today or tomorrow.

 
{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286
1276656637 https://github.com/pydata/xarray/issues/7079#issuecomment-1276656637 https://api.github.com/repos/pydata/xarray/issues/7079 IC_kwDOAMm_X85MGDv9 kthyng 3487237 2022-10-12T19:45:08Z 2022-10-12T19:45:08Z NONE

@ocefpaf and all: thank you! What a mysterious error this has been. Using the workaround

import dask dask.config.set(scheduler="single-threaded")

did indeed avoid the issue for me.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   open_mfdataset parallel=True failing with netcdf4 >= 1.6.1 1385031286

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);