issue_comments

@ocefpaf where you able to make any progress on this? (reminder: NetCDF4 is not installed properly on RTD, with ImportError libmfhdf.so.0: cannot open shared object file: No such file or directory)

If you're busy, I could try to do some trials by adding some other packages to the configuration. This is quite silly, but for some reason I have no such problems with netCDF4 on my salem RTD builds, with a similar config (the only difference I see is the higher number of packages: https://github.com/fmaussion/salem/blob/master/docs/environment.yml)

We are talking about two different issues here. I'll open another issue for the rasm file.

Because you have a RuntimeWarning when reading already

I thought I was able to solve the datetime issue by updating netCDF4 -- do you have the same version in both environments?

The error is visible here

the latest build logs are available here

thanks!

OK, so this didn't help. I'm happy with leaving netCDF4 out of RTD, but I'm unable to convert the rasm file because of the bug I mentioned above.

I wonder if you can reproduce this?

python import xarray as xr ds = xr.tutorial.load_dataset('rasm') ds.to_netcdf('rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')

I wanted to be sure that IO works well with the scipy engine, but I get this error below. Does that look familiar to you?

``` python import xarray as xr ds = xr.open_dataset('/home/mowglie/Downloads/rasm.nc') ds.Tair.encoding['dtype'] = 'float32' ds.to_netcdf('/home/mowglie/rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')

ValueError Traceback (most recent call last) /home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding) 353 try: --> 354 dataset.dump_to_store(store, sync=sync, encoding=encoding) 355 if isinstance(path, BytesIO):

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/core/dataset.py in dump_to_store(self, store, encoder, sync, encoding) 729 if sync: --> 730 store.sync() 731

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/scipy_.py in sync(self) 147 super(ScipyDataStore, self).sync() --> 148 self.ds.flush() 149

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in flush(self) 388 if hasattr(self, 'mode') and self.mode in 'wa': --> 389 self._write() 390 sync = flush

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write(self) 400 self._write_gatt_array() --> 401 self._write_var_array() 402

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_array(self) 448 for name in variables: --> 449 self._write_var_metadata(name) 450 # Now that we have the metadata, we know the vsize of

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_metadata(self, name) 466 for dimname in var.dimensions: --> 467 dimid = self._dims.index(dimname) 468 self._pack_int(dimid)

ValueError: '2' is not in list

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last) <ipython-input-22-4077b7ef5b8c> in <module>() 2 ds = xr.open_dataset('/home/mowglie/Downloads/rasm.nc') 3 ds.Tair.encoding['dtype'] = 'float32' ----> 4 ds.to_netcdf('/home/mowglie/rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding) 780 from ..backends.api import to_netcdf 781 return to_netcdf(self, path, mode, format=format, group=group, --> 782 engine=engine, encoding=encoding) 783 784 dump = utils.function_alias(to_netcdf, 'dump')

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding) 357 finally: 358 if sync: --> 359 store.close() 360 361 if not sync:

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/scipy_.py in close(self) 149 150 def close(self): --> 151 self.ds.close() 152 153 def exit(self, type, value, tb):

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in close(self) 277 if not self.fp.closed: 278 try: --> 279 self.flush() 280 finally: 281 self.variables = {}

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in flush(self) 387 """ 388 if hasattr(self, 'mode') and self.mode in 'wa': --> 389 self._write() 390 sync = flush 391

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write(self) 399 self._write_dim_array() 400 self._write_gatt_array() --> 401 self._write_var_array() 402 403 def _write_numrecs(self):

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_array(self) 447 # Set the metadata for all variables. 448 for name in variables: --> 449 self._write_var_metadata(name) 450 # Now that we have the metadata, we know the vsize of 451 # each record variable, so we can calculate recsize.

/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_metadata(self, name) 465 self._pack_int(len(var.dimensions)) 466 for dimname in var.dimensions: --> 467 dimid = self._dims.index(dimname) 468 self._pack_int(dimid) 469

ValueError: '2' is not in list ```

Yes, with the following code the file size goes down to 9mb:

python import xarray as xr ds = xr.open_dataset('/home/tt/Downloads/rasm.nc') ds.Tair.encoding['dtype'] = 'float32' ds.to_netcdf('/home/tt/rasm.nc', format='NETCDF3_CLASSIC')

I'm happy to do the conversion if you want. One could compress the file a little bit by changing the data to float32, too