issues

Table actions

2 rows where repo = 13221727, state = "open" and user = 16655388 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

state 1

  • open · 2 ✖

repo 1

  • xarray · 2 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
905476569 MDU6SXNzdWU5MDU0NzY1Njk= 5396 bug with to_netcdf when reading dataset via THREDDS sbiner 16655388 open 0     1 2021-05-28T14:08:12Z 2022-02-03T11:49:29Z   NONE      

I am trying to extract data for a specific station from data observations accessed via THREDDS. At first I got an error but a colleague found a way to make it work. In the code below it works if APPLY_PATCH=True but it crashes if APPLY_PATCH=False.

I do not know if this behaviour is expected and/or linked to the way the data is 'served' via THREDDS but I would never have found the way around it by myself.

``` import xarray as xr

CLEF_STATION = "MONTREAL" NOM_VAR = 'prsn' APPLY_PATCH = False

url_pavics = "http://pavics.ouranos.ca:8083/twitcher/ows/proxy/thredds/dodsC" url_donnees_homo = "/".join((url_pavics, "datasets/station_obs/ECCC_AHCCD_gen2_precipitation.ncml"))

ds = xr.open_dataset(url_donnees_homo)

data for specific station(s)

dsr = ds.sel(station=ds.station_name.str.contains(CLEF_STATION))

apply patch

if APPLY_PATCH: for var in dsr.variables: if dsr[var].dtype == 'O': dsr[var].encoding = {}

save to a netcdf file

dsr.to_netcdf("out.nc", format="NETCDF4_CLASSIC") ```

What happened:

When running this I get the following error

``` runfile('/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code/issue_xaray.py', wdir='/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code') Traceback (most recent call last):

File "src/netCDF4/_netCDF4.pyx", line 4931, in netCDF4._netCDF4.Variable.setitem

ValueError: cannot reshape array of size 7 into shape (1,1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/sbiner/cloud/OneDrive - Ouranos inc/job/stagiaires_etudiants/2021_tempete_neige/code/issue_xaray.py", line 14, in <module> dsr.to_netcdf("out.nc", format="NETCDF4_CLASSIC")

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/core/dataset.py", line 1689, in to_netcdf return to_netcdf(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/api.py", line 1107, in to_netcdf dump_to_store(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/api.py", line 1154, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 255, in store self.set_variables(

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 297, in set_variables writer.add(source, target)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/common.py", line 146, in add target[...] = source

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 64, in setitem data[key] = value

File "src/netCDF4/_netCDF4.pyx", line 4933, in netCDF4._netCDF4.Variable.setitem

File "<array_function internals>", line 5, in broadcast_to

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 411, in broadcast_to return _broadcast_to(array, shape, subok=subok, readonly=True)

File "/Users/sbiner/anaconda3/envs/py38/lib/python3.8/site-packages/numpy/lib/stride_tricks.py", line 348, in _broadcast_to it = np.nditer(

ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (1,7) and requested shape (1,1) ``` What you expected to happen:

I expect a netcdf file to be written

Environment:

Output of <tt>xr.show_versions()</tt> INSTALLED VERSIONS ------------------ commit: None python: 3.8.8 (default, Apr 13 2021, 12:59:45) [Clang 10.0.0 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: fr_CA.UTF-8 LANG: fr_CA.UTF-8 LOCALE: fr_CA.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.6.1 xarray: 0.17.0 pandas: 1.2.4 numpy: 1.20.1 scipy: 1.6.2 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.04.0 distributed: 2021.04.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: 0.11.1 numbagg: None pint: None setuptools: 52.0.0.post20210125 pip: 20.3.3 conda: None pytest: None IPython: 7.22.0 sphinx: 4.0.1 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5396/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue
363299007 MDU6SXNzdWUzNjMyOTkwMDc= 2436 save "encoding" when using open_mfdataset sbiner 16655388 open 0     15 2018-09-24T20:02:49Z 2021-07-28T10:35:42Z   NONE      

I like the automatic decoding of the time variable when reading netcdf files but I often need to keep the calendar attribute of the time variable.

Could it be possible to keep those attributes in the DataSet/DataArray return by open_dataset?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/2436/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);