issues: 1203830418
This data as json
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | active_lock_reason | draft | pull_request | body | reactions | performed_via_github_app | state_reason | repo | type |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1203830418 | I_kwDOAMm_X85HwP6S | 6483 | Use of fill_value results in MergeError with otherwise mergable Datasets | 14065102 | open | 0 | 0 | 2022-04-13T21:58:07Z | 2022-04-14T22:07:04Z | NONE | What happened?In What did you expect to happen?I would expect that Minimal Complete Verifiable Example```Python import xarray as xr setupds1 = xr.Dataset(dict( sample_id=("samples", ["sample1"]), sample_value=("samples", [0]), other_array=("others", [0,1,2]), )) ds1 = ds1.assign_coords(dict(samples=ds1.sample_id)) ds2 = xr.Dataset(dict( sample_id=("samples", ["sample2"]), sample_value=("samples", [1]), other_array=("others", [0,1,2]), )) ds2 = ds2.assign_coords(dict(samples=ds2.sample_id)) default merge behavior is correct but alters dtypesxr.merge([ds1, ds2]) <xarray.Dataset>Dimensions: (samples: 2, others: 3)Coordinates:* samples (samples) <U7 'sample1' 'sample2'Dimensions without coordinates: othersData variables:sample_id (samples) object 'sample1' 'sample2'sample_value (samples) float64 0.0 1.0other_array (others) int64 0 1 2using fill_value for either array results in MergeErrorxr.merge( [ds1, ds2], fill_value=dict( sample_id="", sample_value=-1, ) ) MergeError: conflicting values for variable 'sample_id' on objects to be combined. You can skip this check by specifying compat='override'using fill_value with compat="override" results in overriding values of ds2xr.merge( [ds1, ds2], compat="override", fill_value=dict( sample_id="", sample_value=-1, ) ) <xarray.Dataset>Dimensions: (samples: 2, others: 3)Coordinates:* samples (samples) <U7 'sample1' 'sample2'Dimensions without coordinates: othersData variables:sample_id (samples) <U7 'sample1' ''sample_value (samples) int64 0 -1other_array (others) int64 0 1 2``` Relevant log outputTraceback for MergeError ``` ----> 1 xr.merge( 2 [ds1, ds2], 3 fill_value=dict( 4 sample_id="", 5 sample_value=-1, 6 ) 7 ) File ~/miniconda3/envs/sgkit/lib/python3.9/site-packages/xarray/core/merge.py:900, in merge(objects, compat, join, fill_value, combine_attrs) 897 obj = obj.to_dataset(promote_attrs=True) if isinstance(obj, DataArray) else obj 898 dict_like_objects.append(obj) --> 900 merge_result = merge_core( 901 dict_like_objects, 902 compat, 903 join, 904 combine_attrs=combine_attrs, 905 fill_value=fill_value, 906 ) 907 return Dataset._construct_direct(**merge_result._asdict()) File ~/miniconda3/envs/sgkit/lib/python3.9/site-packages/xarray/core/merge.py:635, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value) 632 collected = collect_variables_and_indexes(aligned) 634 prioritized = _get_priority_vars_and_indexes(aligned, priority_arg, compat=compat) --> 635 variables, out_indexes = merge_collected( 636 collected, prioritized, compat=compat, combine_attrs=combine_attrs 637 ) 638 assert_unique_multiindex_level_names(variables) 640 dims = calculate_dimensions(variables) File ~/miniconda3/envs/sgkit/lib/python3.9/site-packages/xarray/core/merge.py:237, in merge_collected(grouped, prioritized, compat, combine_attrs) 235 variables = [variable for variable, _ in elements_list] 236 try: --> 237 merged_vars[name] = unique_variable(name, variables, compat) 238 except MergeError: 239 if compat != "minimal": 240 # we need more than "minimal" compatibility (for which 241 # we drop conflicting coordinates) File ~/miniconda3/envs/sgkit/lib/python3.9/site-packages/xarray/core/merge.py:148, in unique_variable(name, variables, compat, equals) 145 break 147 if not equals: --> 148 raise MergeError( 149 f"conflicting values for variable {name!r} on objects to be combined. " 150 "You can skip this check by specifying compat='override'." 151 ) 153 if combine_method: 154 for var in variables[1:]: MergeError: conflicting values for variable 'sample_id' on objects to be combined. You can skip this check by specifying compat='override'. ``` Anything else we need to know?No response Environment``` INSTALLED VERSIONS commit: None python: 3.9.12 (main, Apr 5 2022, 06:56:58) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 5.4.72-microsoft-standard-WSL2 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.10.3 cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.01.0 distributed: 2022.01.0 matplotlib: 3.5.1 cartopy: None seaborn: 0.11.2 numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 58.0.4 pip: 21.2.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: 4.2.0 ``` |
{ "url": "https://api.github.com/repos/pydata/xarray/issues/6483/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
13221727 | issue |