issues: 1751153826

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
1751153826 I_kwDOAMm_X85oYHyi 7906 `open_dataset` substantially slower with libnetcdf 4.9.2 22833757 closed 0     3 2023-06-10T23:22:08Z 2023-09-13T10:15:56Z 2023-09-13T10:15:56Z NONE      

What is your issue?

Opening datasets is substantially slower with libnetcdf 4.9.2 than with 4.9.1. Oddly, loading netcdf files using the netCDF4 package directly is not affected by the change in libnetcdf version

See times below for loading 10 netcdf files with different package versions:

xarray=2023.5.0, netcdf4=1.6.4, libnetcdf=4.9.2: xarray.open_dataset: 51.814434s netCDF4.Dataset: 1.370544s

xarray=2023.5.0, netcdf4=1.6.3, libnetcdf=4.9.2: xarray.open_dataset: 47.78837s netCDF4.Dataset: 1.821116s

xarray=2023.5.0, netcdf4=1.6.3, libnetcdf=4.9.1: xarray.open_dataset: 2.896161s netCDF4.Dataset: 1.43984s

Script used to get loading times: ``` import pathlib from datetime import datetime import xarray as xr from netCDF4 import Dataset

data_path = pathlib.Path('../data/GOES16') files = list(data_path.rglob("MCMIP.nc"))[:10]

Test xarray

start_time = datetime.now() for file in files: _ = xr.open_dataset(file).CMI_C13.to_numpy() print("xarray.open_dataset:", (datetime.now()-start_time).total_seconds())

Test netCDF4

start_time = datetime.now() for file in files: _ = Dataset(file)["CMI_C13"][:] print("netCDF4.Dataset:", (datetime.now()-start_time).total_seconds()) ```

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/7906/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed 13221727 issue

Links from other tables