home / github / issues

Menu
  • GraphQL API
  • Search all tables

issues: 166511736

This data as json

id node_id number title user state locked assignee milestone comments created_at updated_at closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
166511736 MDU6SXNzdWUxNjY1MTE3MzY= 911 KeyError on saving to NetCDF - due to objects in attrs? 296686 closed 0     3 2016-07-20T07:09:38Z 2016-09-02T22:52:04Z 2016-09-02T22:52:04Z CONTRIBUTOR      

I have an xarray.Dataset that I'm trying to save out to a NetCDF file. The dataset looks like this:

``` python

Out[97]: <xarray.Dataset> Dimensions: (x: 1240, y: 1162) Coordinates: * x (x) float64 -9.476e+05 -9.464e+05 -9.451e+05 -9.439e+05 ... * y (y) float64 1.429e+06 1.428e+06 1.427e+06 1.426e+06 1.424e+06 ... Data variables: data (y, x) float32 nan nan nan nan nan nan nan nan nan nan nan nan ... ```

It has two attributes, both of which have a string key, and a value which is an object (in this case, instances of classes from the rasterio library).

python OrderedDict([('affine', Affine(1256.5430440955893, 0.0, -947639.6305106478, 0.0, -1256.5430440955893, 1429277.8120091767)), ('crs', CRS({'init': 'epsg:27700'}))])

When I try to save out the NetCDF using this code:

python ds.to_netcdf('test.nc')

I get the following error:

```

KeyError Traceback (most recent call last) <ipython-input-96-51e5ff396887> in <module>() ----> 1 ds.to_netcdf('blah3.nc')

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding) 789 from ..backends.api import to_netcdf 790 return to_netcdf(self, path, mode, format=format, group=group, --> 791 engine=engine, encoding=encoding) 792 793 dump = utils.function_alias(to_netcdf, 'dump')

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding) 354 store = store_cls(path, mode, format, group, writer) 355 try: --> 356 dataset.dump_to_store(store, sync=sync, encoding=encoding) 357 if isinstance(path, BytesIO): 358 return path.getvalue()

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/core/dataset.py in dump_to_store(self, store, encoder, sync, encoding) 735 variables, attrs = encoder(variables, attrs) 736 --> 737 store.store(variables, attrs, check_encoding) 738 if sync: 739 store.sync()

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py in store(self, variables, attributes, check_encoding_set) 226 cf_variables, cf_attrs = cf_encoder(variables, attributes) 227 AbstractWritableDataStore.store(self, cf_variables, cf_attrs, --> 228 check_encoding_set)

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py in store(self, variables, attributes, check_encoding_set) 201 if not (k in neccesary_dims and 202 is_trivial_index(v))) --> 203 self.set_variables(variables, check_encoding_set) 204 205 def set_attributes(self, attributes):

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py in set_variables(self, variables, check_encoding_set) 211 name = _encode_variable_name(vn) 212 check = vn in check_encoding_set --> 213 target, source = self.prepare_variable(name, v, check) 214 self.writer.add(source, target) 215

/Users/robin/anaconda3/lib/python3.5/site-packages/xarray/backends/netCDF4_.py in prepare_variable(self, name, variable, check_encoding) 277 # set attributes one-by-one since netCDF4<1.0.10 can't handle 278 # OrderedDict as the input to setncatts --> 279 nc4_var.setncattr(k, v) 280 return nc4_var, variable.data 281

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.setncattr (netCDF4/_netCDF4.c:33460)()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4._set_att (netCDF4/_netCDF4.c:6171)()

/Users/robin/anaconda3/lib/python3.5/collections/init.py in getitem(self, key) 967 if hasattr(self.class, "missing"): 968 return self.class.missing(self, key) --> 969 raise KeyError(key) 970 def setitem(self, key, item): self.data[key] = item 971 def delitem(self, key): del self.data[key]

KeyError: 0 ```

The error seems slightly strange to me, but it seems to be related to saving attributes. If I change the attributes to make all of the values strings (for example, using ds['data'].attrs = {k: repr(v) for k, v in ds['data'].attrs.items()}) then it saves out fine.

Is there a restriction on what sort of values can be stored in attrs and saved out to NetCDF? If so, should this be enforced somehow? It would be ideal if any object could be stored as an attr and saved out (eg. as a pickle) - but this may be difficult (for example, for multiple python versions, if using pickle).

Any thoughts?

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/911/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed 13221727 issue

Links from other tables

  • 0 rows from issues_id in issues_labels
  • 3 rows from issue in issue_comments
Powered by Datasette · Queries took 0.646ms · About: xarray-datasette