home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 457874038

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
457874038 MDU6SXNzdWU0NTc4NzQwMzg= 3034 open_mfdataset fails on variable attributes with 'list' type 11731042 closed 0     6 2019-06-19T08:25:23Z 2019-08-04T17:41:32Z 2019-08-04T17:41:32Z NONE      

Using open_mfdataset on a series of netcdf files having variable attributes with type list will fail with the following exception, when these attributes have different values from one file to another:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

ncf = xarray.open_mfdataset(files) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/backends/api.py", line 658, in open_mfdataset ids=ids) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 553, in _auto_combine data_vars=data_vars, coords=coords) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 474, in _combine_nd compat=compat) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 492, in _auto_combine_all_along_first_dim data_vars, coords) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 510, in _auto_combine_1d for id, ds_group in grouped_by_vars] File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 368, in _auto_concat return concat(datasets, dim=dim, data_vars=data_vars, coords=coords) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 122, in concat return f(objs, dim, data_vars, coords, compat, positions) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/combine.py", line 307, in _dataset_concat combined = concat_vars(vars, dim, positions) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/variable.py", line 1982, in concat return Variable.concat(variables, dim, positions, shortcut) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/variable.py", line 1433, in concat utils.remove_incompatible_items(attrs, var.attrs) File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/utils.py", line 184, in remove_incompatible_items not compat(first_dict[k], second_dict[k]))): File "/home/ananda/jfpiolle/miniconda2/envs/cerbere/lib/python2.7/site-packages/xarray/core/utils.py", line 133, in equivalent (pd.isnull(first) and pd.isnull(second))) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

An example of such variable is provided below:

double sea_ice_fraction(time) ; sea_ice_fraction:least_significant_digit = 2LL ; sea_ice_fraction:_FillValue = 1.e+20 ; sea_ice_fraction:long_name = "sea ice fraction" ; sea_ice_fraction:standard_name = "sea_ice_fraction" ; sea_ice_fraction:authority = "CF 1.7" ; sea_ice_fraction:units = "1" ; sea_ice_fraction:coverage_content_type = "auxiliaryInformation" ; sea_ice_fraction:coordinates = "time lon lat" ; sea_ice_fraction:source = "CCI Sea Ice" ; sea_ice_fraction:institution = "ESA" ; string sea_ice_fraction:source_files = "ice_conc_nh_ease2-250_cdr-v2p0_199912011200.nc", "ice_conc_sh_ease2-250_cdr-v2p0_199912011200.nc" ;

The exception will occur when the source_files attribute have a different values in the file time series I am trying to concatenate. I had to use the preprocess argument to remove first this attribute to avoid this exception.

This is caused by the equivalent method in xarray/core/utils.py that does not account for this case:

python def equivalent(first, second): """Compare two objects for equivalence (identity or equality), using array_equiv if either object is an ndarray """ # TODO: refactor to avoid circular import from . import duck_array_ops if isinstance(first, np.ndarray) or isinstance(second, np.ndarray): return duck_array_ops.array_equiv(first, second) else: return ((first is second) or (first == second) or (pd.isnull(first) and pd.isnull(second)))

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3034/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 2 rows from issues_id in issues_labels
  • 6 rows from issue in issue_comments
Powered by Datasette · Queries took 0.912ms · About: xarray-datasette