home / github / issues

Menu
  • Search all tables
  • GraphQL API

issues: 905476569

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
905476569 MDU6SXNzdWU5MDU0NzY1Njk= 5396 bug with to_netcdf when reading dataset via THREDDS 16655388 open 0     1 2021-05-28T14:08:12Z 2022-02-03T11:49:29Z   NONE      

I am trying to extract data for a specific station from data observations accessed via THREDDS. At first I got an error but a colleague found a way to make it work. In the code below it works if APPLY_PATCH=True but it crashes if APPLY_PATCH=False.

I do not know if this behaviour is expected and/or linked to the way the data is 'served' via THREDDS but I would never have found the way around it by myself.

``` import xarray as xr

CLEF_STATION = "MONTREAL" NOM_VAR = 'prsn' APPLY_PATCH = False

url_pavics = "http://pavics.ouranos.ca:8083/twitcher/ows/proxy/thredds/dodsC" url_donnees_homo = "/".join((url_pavics, "datasets/station_obs/ECCC_AHCCD_gen2_precipitation.ncml"))

ds = xr.open_dataset(url_donnees_homo)

data for specific station(s)

dsr = ds.sel(station=ds.station_name.str.contains(CLEF_STATION))

apply patch

if APPLY_PATCH: for var in dsr.variables: if dsr[var].dtype == 'O': dsr[var].encoding = {}

save to a netcdf file

dsr.to_netcdf("out.nc", format="NETCDF4_CLASSIC") ```

What happened:

When running this I get the following error

``` runfile('/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code/issue_xaray.py', wdir='/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code') Traceback (most recent call last):

File "src/netCDF4/_netCDF4.pyx", line 4931, in netCDF4._netCDF4.Variable.setitem

ValueError: cannot reshape array of size 7 into shape (1,1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code/issue_xaray.py", line 14, in <module> dsr.to_netcdf("out.nc", format="NETCDF4_CLASSIC")

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/core/dataset.py", line 1689, in to_netcdf return to_netcdf(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/api.py", line 1107, in to_netcdf dump_to_store(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/api.py", line 1154, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 255, in store self.set_variables(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 297, in set_variables writer.add(source, target)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 146, in add target[...] = source

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 64, in setitem data[key] = value

File "src/netCDF4/_netCDF4.pyx", line 4933, in netCDF4._netCDF4.Variable.setitem

File "<array_function internals>", line 5, in broadcast_to

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 411, in broadcast_to return _broadcast_to(array, shape, subok=subok, readonly=True)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 348, in _broadcast_to it = np.nditer(

ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (1,7) and requested shape (1,1) ``` What you expected to happen:

I expect a netcdf file to be written

Environment:

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.8.8 (default, Apr 13 2021, 12:59:45) [Clang 10.0.0 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: fr_CA.UTF-8 LANG: fr_CA.UTF-8 LOCALE: fr_CA.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.6.1 xarray: 0.17.0 pandas: 1.2.4 numpy: 1.20.1 scipy: 1.6.2 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.04.0 distributed: 2021.04.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: 0.11.1 numbagg: None pint: None setuptools: 52.0.0.post20210125 pip: 20.3.3 conda: None pytest: None IPython: 7.22.0 sphinx: 4.0.1
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5396/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 1 row from issue in issue_comments
Powered by Datasette · Queries took 0.7ms · About: xarray-datasette