issues

Table actions

3 rows where repo = 13221727, state = "open" and user = 22542812 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

state 1

  • open · 3 ✖

repo 1

  • xarray · 3 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
491215043 MDU6SXNzdWU0OTEyMTUwNDM= 3297 Add writing complex data to docs DerWeh 22542812 open 0     10 2019-09-09T17:01:45Z 2023-08-11T23:55:56Z   NONE      

Is there a recommended way how to save complex data? I found some option on stack overflow, but they don't seem to satisfactory.

The main point of having self-describing data which I write as binary data, is that people can just read the data, and don't have to worry how to interpret it. Thus, the only viable option to me would be using engine='h5netcdf'.

On the other hand, if something like adding an axis would be done internally by xarray it would be also OK, as everyone could read my data using the library.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3297/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue
874695249 MDU6SXNzdWU4NzQ2OTUyNDk= 5254 Boolean confusion DerWeh 22542812 open 0     3 2021-05-03T15:53:54Z 2021-05-05T06:50:21Z   NONE      

True and True are treated different depending on whether it is a bool or a np.bool_. This creates absolute hellish errors. You save date, and load it again. Then bool turned to np.bool_ and things work differently. I.e. you cannot save your data again.

The following example works perfectly fine: ```python

data = xr.Dataset() data.attrs['bool_type'] = True data.to_netcdf() data <xarray.Dataset> Dimensions: () Data variables: empty Attributes: bool_type: True The following example however fails:python data = xr.Dataset() data.attrs['bool_type'] = np.True_ data.to_netcdf() TypeError: Invalid value for attr: True must be a number, a string, an ndarray or a list/tuple of numbers/strings for serialization to netCDF files data <xarray.Dataset> Dimensions: () Data variables: empty Attributes: bool_type: True ```

If you load the data again, in both cases np.True_ leading to confusing things, when the seemingly same things fails.

For this routine a simple remedy is probably expanding the instance check in check_attr:

python isinstance(value, (str, Number, np.ndarray, np.number, np.bool_, list, tuple))

As numpy warns, np.bool_ are no numbers in contrast to regular Python, see https://numpy.org/doc/stable/reference/arrays.scalars.html#numpy.bool_.

I fear, however, that this problem runs deeper and the same issue might arise at different points.

edit: included TypeError

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/5254/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue
720315478 MDU6SXNzdWU3MjAzMTU0Nzg= 4507 Dropping of unaligned Data at assignment to Dataset DerWeh 22542812 open 0     2 2020-10-13T14:13:28Z 2020-10-13T14:59:37Z   NONE      

What happened: I recently ran into the trouble as I assigned data generate by an external program to a dataset, and suddenly the dataset contained only NaN, see the example below. The issue was, that the program rounded numbers to 10 digits, so the coordinates didn't match anymore. xarray silently ignores this.

What you expected to happen: I would have expected an error or at least a warning, when the coordinates don't match. The current behavior can lead to bugs which are very hard to trace.

Minimal Complete Verifiable Example:

```python import numpy as np import xarray as xr

x = np.linspace(0, 1) dataset = xr.Dataset(coords={'x': x}) data = xr.DataArray(np.random.random(50), dims=['x'], coords={'x': np.around(x, decimals=10)})

dataset['data'] = data print(dataset.data) print(dataset.coords['x']) print(data.coords['x']) Output:python

print(dataset.data)

<xarray.DataArray 'data' (x: 50)> array([0.20134419, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 0.98357925]) Coordinates: * x (x) float64 0.0 0.02041 0.04082 0.06122 ... 0.9592 0.9796 1.0

print(dataset.coords['x'])

<xarray.DataArray 'x' (x: 50)> array([0. , 0.020408, 0.040816, 0.061224, 0.081633, 0.102041, 0.122449, 0.142857, 0.163265, 0.183673, 0.204082, 0.22449 , 0.244898, 0.265306, 0.285714, 0.306122, 0.326531, 0.346939, 0.367347, 0.387755, 0.408163, 0.428571, 0.44898 , 0.469388, 0.489796, 0.510204, 0.530612, 0.55102 , 0.571429, 0.591837, 0.612245, 0.632653, 0.653061, 0.673469, 0.693878, 0.714286, 0.734694, 0.755102, 0.77551 , 0.795918, 0.816327, 0.836735, 0.857143, 0.877551, 0.897959, 0.918367, 0.938776, 0.959184, 0.979592, 1. ]) Coordinates: * x (x) float64 0.0 0.02041 0.04082 0.06122 ... 0.9592 0.9796 1.0

print(data.coords['x'])

<xarray.DataArray 'x' (x: 50)> array([0. , 0.020408, 0.040816, 0.061224, 0.081633, 0.102041, 0.122449, 0.142857, 0.163265, 0.183673, 0.204082, 0.22449 , 0.244898, 0.265306, 0.285714, 0.306122, 0.326531, 0.346939, 0.367347, 0.387755, 0.408163, 0.428571, 0.44898 , 0.469388, 0.489796, 0.510204, 0.530612, 0.55102 , 0.571429, 0.591837, 0.612245, 0.632653, 0.653061, 0.673469, 0.693878, 0.714286, 0.734694, 0.755102, 0.77551 , 0.795918, 0.816327, 0.836735, 0.857143, 0.877551, 0.897959, 0.918367, 0.938776, 0.959184, 0.979592, 1. ]) Coordinates: * x (x) float64 0.0 0.02041 0.04082 0.06122 ... 0.9592 0.9796 1.0

``` Anything else we need to know?:

Environment:

Output of <tt>xr.show_versions()</tt> ```sh $ py -c "import xarray as xr; xr.show_versions()" INSTALLED VERSIONS ------------------ commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-118-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.6.1 xarray: 0.16.1 pandas: 1.0.5 numpy: 1.18.5 scipy: 1.5.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.8.1 h5py: 2.9.0 Nio: None zarr: None cftime: 1.1.2 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.13.0 distributed: 2.13.0 matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None pint: None setuptools: 46.1.3 pip: 19.3.1 conda: 4.8.5 pytest: 5.1.2 IPython: 7.18.1 sphinx: 3.0.2 ``` 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4507/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
     xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);