home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 657798184

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2995#issuecomment-657798184 https://api.github.com/repos/pydata/xarray/issues/2995 657798184 MDEyOklzc3VlQ29tbWVudDY1Nzc5ODE4NA== 2067093 2020-07-13T21:17:06Z 2020-07-13T21:17:06Z NONE

I ran into this issue, here's a simple workaround that seems to work:

```python def dataset_to_bytes(ds: xr.Dataset, name: str = "my-dataset") -> bytes: """Converts datset to bytes."""

nc4_ds = netCDF4.Dataset(name, mode="w", diskless=True, memory=ds.nbytes)
nc4_store = NetCDF4DataStore(nc4_ds)
dump_to_store(ds, nc4_store)
res_mem = nc4_ds.close()
res_bytes = res_mem.tobytes()
return res_bytes

```

I tested this using the following:

```python import BytesIO

fname = "REDACTED.nc" ds = xr.load_dataset(fname) ds_bytes = dataset_to_bytes(ds) ds2 = xr.load_dataset(BytesIO(ds_bytes))

assert ds2.equals(ds) and all(ds2.attrs[k]==ds.attrs[k] for k in set(ds2.attrs).union(ds.attrs)) ```

The assertion holds true, however the file size on disk is different. It's possible they were saved using different netCDF4 versions, I haven't had time to test that.

I tried using just ds.to_netcdf() but get the following error:

`ValueError: NetCDF 3 does not support type |S32`

That's because it falls back to the 'scipy' engine. Would be nice to have a non-hacky way to write netcdf4 files to byte streams. :smiley:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  449706080
Powered by Datasette · Queries took 0.677ms · About: xarray-datasette