issue_comments

Table actions

6 rows where issue = 1323734180 and user = 302469 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

  • QuLogic · 6 ✖

issue 1

  • test_open_nczarr uses too much memory · 6 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1538068408 https://github.com/pydata/xarray/issues/6854#issuecomment-1538068408 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85brQ-4 QuLogic 302469 2023-05-08T09:42:08Z 2023-05-08T09:42:08Z CONTRIBUTOR 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180
1382625582 https://github.com/pydata/xarray/issues/6854#issuecomment-1382625582 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85SaTEu QuLogic 302469 2023-01-14T01:33:08Z 2023-01-14T01:33:08Z CONTRIBUTOR

Building against netcdf with the patch backported by @opoplawski, I now get a single failure: ``` ____ TestNCZarr.testopen_nczarr ______ [gw3] linux -- Python 3.11.1 /usr/bin/python3 zarr_obj = <zarr.core.Array '/dim2' (9,) float64 read-only> dimension_key = '_ARRAY_DIMENSIONS', try_nczarr = True def _get_zarr_dims_and_attrs(zarr_obj, dimension_key, try_nczarr): # Zarr arrays do not have dimensions. To get around this problem, we add # an attribute that specifies the dimension. We have to hide this attribute # when we send the attributes to the user. # zarr_obj can be either a zarr group or zarr array try: # Xarray-Zarr

      dimensions = zarr_obj.attrs[dimension_key]

/builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:183:

self = <zarr.attrs.Attributes object at 0x7f697d934ad0> item = '_ARRAY_DIMENSIONS' def getitem(self, item):

  return self.asdict()[item]

E KeyError: '_ARRAY_DIMENSIONS' /usr/lib/python3.11/site-packages/zarr/attrs.py:74: KeyError During handling of the above exception, another exception occurred: zarr_obj = <zarr.core.Array '/dim2' (9,) float64 read-only> dimension_key = '_ARRAY_DIMENSIONS', try_nczarr = True def _get_zarr_dims_and_attrs(zarr_obj, dimension_key, try_nczarr): # Zarr arrays do not have dimensions. To get around this problem, we add # an attribute that specifies the dimension. We have to hide this attribute # when we send the attributes to the user. # zarr_obj can be either a zarr group or zarr array try: # Xarray-Zarr dimensions = zarr_obj.attrs[dimension_key] except KeyError as e: if not try_nczarr: raise KeyError( f"Zarr object is missing the attribute {dimension_key}, which is " "required for xarray to determine variable dimensions." ) from e

        # NCZarr defines dimensions through metadata in .zarray
        zarray_path = os.path.join(zarr_obj.path, ".zarray")
        zarray = json.loads(zarr_obj.store[zarray_path])
        try:
            # NCZarr uses Fully Qualified Names
            dimensions = [

              os.path.basename(dim) for dim in zarray["_NCZARR_ARRAY"]["dimrefs"]
            ]

E KeyError: '_NCZARR_ARRAY' /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:197: KeyError The above exception was the direct cause of the following exception: self = <xarray.tests.test_backends.TestNCZarr object at 0x7f697ed62f10> def test_open_nczarr(self) -> None: with create_tmp_file(suffix=".zarr") as tmp: expected = self._create_nczarr(tmp) actual = xr.open_zarr(tmp, consolidated=False) /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/tests/test_backends.py:5741:

/builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:819: in open_zarr ds = open_dataset( /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/api.py:540: in open_dataset backend_ds = backend.open_dataset( /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:897: in open_dataset ds = store_entrypoint.open_dataset( /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/store.py:28: in open_dataset vars, attrs = store.load() /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/common.py:128: in load (_decode_variable_name(k), v) for k, v in self.get_variables().items() /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:480: in get_variables return FrozenDict( /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/core/utils.py:469: in FrozenDict return Frozen(dict(args, *kwargs)) /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:481: in <genexpr> (k, self.open_store_variable(k, v)) for k, v in self.zarr_group.arrays() /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:457: in open_store_variable dimensions, attributes = _get_zarr_dims_and_attrs(

zarr_obj = <zarr.core.Array '/dim2' (9,) float64 read-only> dimension_key = '_ARRAY_DIMENSIONS', try_nczarr = True def _get_zarr_dims_and_attrs(zarr_obj, dimension_key, try_nczarr): # Zarr arrays do not have dimensions. To get around this problem, we add # an attribute that specifies the dimension. We have to hide this attribute # when we send the attributes to the user. # zarr_obj can be either a zarr group or zarr array try: # Xarray-Zarr dimensions = zarr_obj.attrs[dimension_key] except KeyError as e: if not try_nczarr: raise KeyError( f"Zarr object is missing the attribute {dimension_key}, which is " "required for xarray to determine variable dimensions." ) from e

        # NCZarr defines dimensions through metadata in .zarray
        zarray_path = os.path.join(zarr_obj.path, ".zarray")
        zarray = json.loads(zarr_obj.store[zarray_path])
        try:
            # NCZarr uses Fully Qualified Names
            dimensions = [
                os.path.basename(dim) for dim in zarray["_NCZARR_ARRAY"]["dimrefs"]
            ]
        except KeyError as e:

          raise KeyError(
                f"Zarr object is missing the attribute `{dimension_key}` and the NCZarr metadata, "
                "which are required for xarray to determine variable dimensions."
            ) from e

E KeyError: 'Zarr object is missing the attribute _ARRAY_DIMENSIONS and the NCZarr metadata, which are required for xarray to determine variable dimensions.' /builddir/build/BUILDROOT/python-xarray-2022.12.0-1.fc38.x86_64/usr/lib/python3.11/site-packages/xarray/backends/zarr.py:200: KeyError ```

Does that look like a problem in netcdf or a problem in xarray here?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180
1330143162 https://github.com/pydata/xarray/issues/6854#issuecomment-1330143162 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85PSF-6 QuLogic 302469 2022-11-29T06:21:17Z 2022-11-29T06:21:17Z CONTRIBUTOR

I could, but I'm not sure what to try. I tried running https://github.com/Unidata/netcdf4-python/blob/master/examples/tutorial.py with those versions and that worked without issue.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180
1328135253 https://github.com/pydata/xarray/issues/6854#issuecomment-1328135253 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85PKbxV QuLogic 302469 2022-11-26T23:57:28Z 2022-11-26T23:57:28Z CONTRIBUTOR

Still broken in 2022.11.0; any ideas what to look at here?

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180
1200757563 https://github.com/pydata/xarray/issues/6854#issuecomment-1200757563 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85Hkhs7 QuLogic 302469 2022-08-01T06:14:42Z 2022-08-01T06:14:42Z CONTRIBUTOR

Backporting netcdf 4.9.0 to Fedora 36 also OOMs, so there's something weird there, but not sure if it's netCDF, or something that xarray is doing.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180
1200657104 https://github.com/pydata/xarray/issues/6854#issuecomment-1200657104 https://api.github.com/repos/pydata/xarray/issues/6854 IC_kwDOAMm_X85HkJLQ QuLogic 302469 2022-08-01T03:35:39Z 2022-08-01T03:35:39Z CONTRIBUTOR

Hmm, I seem to be able to build on Fedora 36 (instead of Rawhide above), with the following versions:

INSTALLED VERSIONS ------------------ commit: None python: 3.10.5 (main, Jun 9 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)] python-bits: 64 OS: Linux OS-release: 5.17.13-300.fc36.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.3.5 numpy: 1.22.0 scipy: 1.8.1 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.5.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2022.05.0 distributed: None matplotlib: 3.5.2 cartopy: None seaborn: 0.11.1 numbagg: None fsspec: 2022.5.0 cupy: None pint: 0.16.1 sparse: None flox: None numpy_groupies: None setuptools: 59.6.0 pip: 21.3.1 conda: None pytest: 6.2.5 IPython: None sphinx: 4.4.0 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   test_open_nczarr uses too much memory 1323734180

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);