home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 397063221

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
397063221 MDU6SXNzdWUzOTcwNjMyMjE= 2662 open_mfdataset in v.0.11.1 is very slow 22245117 closed 0     6 2019-01-08T19:59:47Z 2019-01-17T13:05:43Z 2019-01-17T13:05:43Z CONTRIBUTOR      

I have several repositories corresponding to different time periods of time. Each repository contains several netCDF files with different variables. Here is a simplified example: path/period1/ contains varsABC_period1.nc and varsDEF_period1.nc, and path/period2/ contains varsABC_period2.nc and varsDEF_period2.nc. All variables have the dimension T (time), but the other dimensions can be different (e.g., A has dimensions [T,X1,Y1], B has dimensions [T,X1,Y2], ...). Before version v0.11.1 a was able to easily open a dataset as follow: ```python ds=xr.open_mfdataset('path\period**.nc', concat_dim='T')

However, `open_mfdataset` now fails. It actually takes forever without returning any output, so my guess is that it's just very very slow. I've noticed that I'm able to create the dataset concatenating the files with the same variables first, then merging them:python dsABC=xr.open_mfdataset('path\period\varsABC.nc', concat_dim='T') dsDEF=xr.open_mfdataset('path\period\varsDEF.nc', concat_dim='T') ds=xr.merge([dsABC, dsDEF]) ``` I can't find what's the update in v0.11.1 that is causing this behavior. Any idea?

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.6 | packaged by conda-forge | (default, Oct 12 2018, 14:08:43) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] python-bits: 64 OS: Linux OS-release: 3.10.0-862.3.2.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.11.2 pandas: 0.23.4 numpy: 1.14.3 scipy: 1.1.0 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: 1.0.0 distributed: 1.25.2 matplotlib: 3.0.0 cartopy: None seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.1 conda: 4.5.12 pytest: 3.8.2 IPython: 7.0.1 sphinx: 1.8.1
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2662/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 1 row from issues_id in issues_labels
  • 6 rows from issue in issue_comments
Powered by Datasette · Queries took 0.718ms · About: xarray-datasette