html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1106#issuecomment-262408119,https://api.github.com/repos/pydata/xarray/issues/1106,262408119,MDEyOklzc3VlQ29tbWVudDI2MjQwODExOQ==,950575,2016-11-23T00:38:20Z,2016-11-23T00:38:20Z,CONTRIBUTOR,"> @ocefpaf where you able to make any progress on this?
Sorry but no. I will look at it again but I do remember seeing some extra `conda install` commands issued after the env was create, and that may lead to undesirable up-downgrade of packages. If possible, the RTD env should install everything in one go, with a single ` conda env create environment.yml`.
PS: Not sure what your timezone is but due to my ignorance on RTD it would be nice if we could touch base on `gitter` at some point. We can probably figure this out quickly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-262387452,https://api.github.com/repos/pydata/xarray/issues/1106,262387452,MDEyOklzc3VlQ29tbWVudDI2MjM4NzQ1Mg==,10050469,2016-11-22T22:39:25Z,2016-11-22T22:39:25Z,MEMBER,"@ocefpaf where you able to make any progress on this? (reminder: NetCDF4 is not installed properly on RTD, with ImportError ``libmfhdf.so.0: cannot open shared object file: No such file or directory``)
If you're busy, I could try to do some trials by adding some other packages to the configuration. This is quite silly, but for some reason I have no such problems with netCDF4 on my salem RTD builds, with a similar config (the only difference I see is the higher number of packages: https://github.com/fmaussion/salem/blob/master/docs/environment.yml)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260033149,https://api.github.com/repos/pydata/xarray/issues/1106,260033149,MDEyOklzc3VlQ29tbWVudDI2MDAzMzE0OQ==,950575,2016-11-11T19:16:10Z,2016-11-11T19:16:10Z,CONTRIBUTOR,"> We are talking about two different issues here. I'll open another issue for the rasm file.
Ah OK. Sorry for the noise.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260032970,https://api.github.com/repos/pydata/xarray/issues/1106,260032970,MDEyOklzc3VlQ29tbWVudDI2MDAzMjk3MA==,10050469,2016-11-11T19:15:17Z,2016-11-11T19:15:17Z,MEMBER,"We are talking about two different issues here. I'll open another issue for the rasm file.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260030024,https://api.github.com/repos/pydata/xarray/issues/1106,260030024,MDEyOklzc3VlQ29tbWVudDI2MDAzMDAyNA==,950575,2016-11-11T19:02:01Z,2016-11-11T19:10:48Z,CONTRIBUTOR,"> The error is visible here
> the latest build logs are available here
> thanks!
Thanks @fmaussion I can see that there are multiple conda install commands the the subsequent ones change the env. For example:
```
conda install --yes --name latest sphinx==1.3.5 Pygments==2.1.1 docutils==0.12 mock pillow==3.0.0 sphinx_rtd_theme==0.1.7 alabaster>=0.7,<0.8,!=0.7.5
Fetching package metadata: ....
Solving package specifications: .........
Package plan for installation in environment /home/docs/checkouts/readthedocs.org/user_builds/xray/conda/latest:
The following NEW packages will be INSTALLED:
jbig: 2.1-0
sphinx_rtd_theme: 0.1.7-py27_0
The following packages will be DOWNGRADED:
freetype: 2.6.3-1 --> 2.5.5-1
jpeg: 9b-0 --> 8d-2
libtiff: 4.0.6-7 --> 4.0.6-2
pillow: 3.4.2-py27_0 --> 3.0.0-py27_1
pygments: 2.1.3-py27_1 --> 2.1.1-py27_0
sphinx: 1.4.8-py27_0 --> 1.3.5-py27_0
tk: 8.5.19-0 --> 8.5.18-0
```
That `jpeg` change is not desirable!
I also cannot see the full log from `conda env create --name latest --file /home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/latest/doc/environment.yml`.
> It's the same environment, I'm literally typing ""git checkout master"" or
> ""git checkout v0.8.2"" in my xarray-dev conda environment (on Python 3.5)
I am not sure it is due to the change I mention above. I will make a few experiments and report back.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260029785,https://api.github.com/repos/pydata/xarray/issues/1106,260029785,MDEyOklzc3VlQ29tbWVudDI2MDAyOTc4NQ==,1217238,2016-11-11T19:00:52Z,2016-11-11T19:00:52Z,MEMBER,"It's the same environment, I'm literally typing ""git checkout master"" or
""git checkout v0.8.2"" in my xarray-dev conda environment (on Python 3.5)
On Fri, Nov 11, 2016 at 10:59 AM, Fabien Maussion notifications@github.com
wrote:
> Because you have a RuntimeWarning when reading already
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/xarray/issues/1106#issuecomment-260029534, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ABKS1iqw2jD48U-qC5Kjed0Ut3Lz_BS4ks5q9LsjgaJpZM4Ku57C
> .
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260029534,https://api.github.com/repos/pydata/xarray/issues/1106,260029534,MDEyOklzc3VlQ29tbWVudDI2MDAyOTUzNA==,10050469,2016-11-11T18:59:46Z,2016-11-11T18:59:46Z,MEMBER,"Because you have a `RuntimeWarning` when reading already
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260029349,https://api.github.com/repos/pydata/xarray/issues/1106,260029349,MDEyOklzc3VlQ29tbWVudDI2MDAyOTM0OQ==,10050469,2016-11-11T18:58:56Z,2016-11-11T18:58:56Z,MEMBER,"I thought I was able to solve the datetime issue by updating netCDF4 -- do you have the same version in both environments?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260028968,https://api.github.com/repos/pydata/xarray/issues/1106,260028968,MDEyOklzc3VlQ29tbWVudDI2MDAyODk2OA==,1217238,2016-11-11T18:57:17Z,2016-11-11T18:57:17Z,MEMBER,"I get the same error attempting to same RASM on xarray v0.8.2.
On xarray master, I get a different error, related to encoding datetimes:
```
In [1]: import xarray as xr
In [2]: ds = xr.tutorial.load_dataset('rasm')
/Users/shoyer/dev/xarray/xarray/conventions.py:386: RuntimeWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy netCDF4.datetime objects instead, reason: dates out of range
result = decode_cf_datetime(example_value, units, calendar)
In [3]: ds.to_netcdf('rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-f834a013f387> in <module>()
----> 1 ds.to_netcdf('rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')
/Users/shoyer/dev/xarray/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding)
801 from ..backends.api import to_netcdf
802 return to_netcdf(self, path, mode, format=format, group=group,
--> 803 engine=engine, encoding=encoding)
804
805 def __unicode__(self):
/Users/shoyer/dev/xarray/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding)
513 store = store_cls(path, mode, format, group, writer)
514 try:
--> 515 dataset.dump_to_store(store, sync=sync, encoding=encoding)
516 if isinstance(path, BytesIO):
517 return path.getvalue()
/Users/shoyer/dev/xarray/xarray/core/dataset.py in dump_to_store(self, store, encoder, sync, encoding)
747 variables, attrs = encoder(variables, attrs)
748
--> 749 store.store(variables, attrs, check_encoding)
750 if sync:
751 store.sync()
/Users/shoyer/dev/xarray/xarray/backends/common.py in store(self, variables, attributes, check_encoding_set)
230 # All NetCDF files get CF encoded by default, without this attempting
231 # to write times, for example, would fail.
--> 232 cf_variables, cf_attrs = cf_encoder(variables, attributes)
233 AbstractWritableDataStore.store(self, cf_variables, cf_attrs,
234 check_encoding_set)
/Users/shoyer/dev/xarray/xarray/conventions.py in cf_encoder(variables, attributes)
1067 """"""
1068 new_vars = OrderedDict((k, encode_cf_variable(v, name=k))
-> 1069 for k, v in iteritems(variables))
1070 return new_vars, attributes
/Users/shoyer/dev/xarray/xarray/conventions.py in <genexpr>(.0)
1067 """"""
1068 new_vars = OrderedDict((k, encode_cf_variable(v, name=k))
-> 1069 for k, v in iteritems(variables))
1070 return new_vars, attributes
/Users/shoyer/dev/xarray/xarray/conventions.py in encode_cf_variable(var, needs_copy, name)
721 var, needs_copy = maybe_encode_offset_and_scale(var, needs_copy)
722 var, needs_copy = maybe_encode_fill_value(var, needs_copy)
--> 723 var = maybe_encode_dtype(var, name)
724 var = maybe_encode_bools(var)
725 var = ensure_dtype_not_object(var)
/Users/shoyer/dev/xarray/xarray/conventions.py in maybe_encode_dtype(var, name)
634 data = string_to_char(np.asarray(data, 'S'))
635 dims = dims + ('string%s' % data.shape[-1],)
--> 636 data = data.astype(dtype=dtype)
637 var = Variable(dims, data, attrs, encoding)
638 return var
TypeError: float() argument must be a string or a number, not 'datetime.datetime'
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260028121,https://api.github.com/repos/pydata/xarray/issues/1106,260028121,MDEyOklzc3VlQ29tbWVudDI2MDAyODEyMQ==,10050469,2016-11-11T18:53:25Z,2016-11-11T18:53:25Z,MEMBER,"The error is visible [here](http://xarray.pydata.org/en/latest/examples/multidimensional-coords.html)
the latest build logs are available [here](https://readthedocs.org/projects/xray/builds/4651830/)
thanks!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260027475,https://api.github.com/repos/pydata/xarray/issues/1106,260027475,MDEyOklzc3VlQ29tbWVudDI2MDAyNzQ3NQ==,950575,2016-11-11T18:50:37Z,2016-11-11T18:50:37Z,CONTRIBUTOR,"Can you point me to the error and some details on RTD. They may need to update the `conda` version to get it to work.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-260027243,https://api.github.com/repos/pydata/xarray/issues/1106,260027243,MDEyOklzc3VlQ29tbWVudDI2MDAyNzI0Mw==,10050469,2016-11-11T18:49:25Z,2016-11-11T18:49:25Z,MEMBER,"OK, so this didn't help. I'm happy with leaving netCDF4 out of RTD, but I'm unable to convert the rasm file because of the bug I mentioned above.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259823644,https://api.github.com/repos/pydata/xarray/issues/1106,259823644,MDEyOklzc3VlQ29tbWVudDI1OTgyMzY0NA==,950575,2016-11-10T22:15:17Z,2016-11-10T22:15:17Z,CONTRIBUTOR,"@fmaussion I am away from a laptop to test this but the following change should fix it (depending on the conda version you have there) :
``` yaml
channels:
- conda-forge
- defaults
```
By explicitly adding `defaults` below `conda-forge` you will ensure that the right set of packages will be downloaded thanks to the channel preference feature in recent `conda` version.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259821280,https://api.github.com/repos/pydata/xarray/issues/1106,259821280,MDEyOklzc3VlQ29tbWVudDI1OTgyMTI4MA==,10050469,2016-11-10T22:05:04Z,2016-11-10T22:05:04Z,MEMBER,"I wonder if you can reproduce this?
``` python
import xarray as xr
ds = xr.tutorial.load_dataset('rasm')
ds.to_netcdf('rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259803141,https://api.github.com/repos/pydata/xarray/issues/1106,259803141,MDEyOklzc3VlQ29tbWVudDI1OTgwMzE0MQ==,1217238,2016-11-10T20:48:28Z,2016-11-10T20:48:28Z,MEMBER,"I have not seen that one before!
Maybe it's an issue because that file already exists?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259792002,https://api.github.com/repos/pydata/xarray/issues/1106,259792002,MDEyOklzc3VlQ29tbWVudDI1OTc5MjAwMg==,10050469,2016-11-10T20:00:45Z,2016-11-10T20:00:45Z,MEMBER,"I wanted to be sure that IO works well with the scipy engine, but I get this error below. Does that look familiar to you?
``` python
import xarray as xr
ds = xr.open_dataset('/home/mowglie/Downloads/rasm.nc')
ds.Tair.encoding['dtype'] = 'float32'
ds.to_netcdf('/home/mowglie/rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding)
353 try:
--> 354 dataset.dump_to_store(store, sync=sync, encoding=encoding)
355 if isinstance(path, BytesIO):
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/core/dataset.py in dump_to_store(self, store, encoder, sync, encoding)
729 if sync:
--> 730 store.sync()
731
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/scipy_.py in sync(self)
147 super(ScipyDataStore, self).sync()
--> 148 self.ds.flush()
149
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in flush(self)
388 if hasattr(self, 'mode') and self.mode in 'wa':
--> 389 self._write()
390 sync = flush
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write(self)
400 self._write_gatt_array()
--> 401 self._write_var_array()
402
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_array(self)
448 for name in variables:
--> 449 self._write_var_metadata(name)
450 # Now that we have the metadata, we know the vsize of
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_metadata(self, name)
466 for dimname in var.dimensions:
--> 467 dimid = self._dims.index(dimname)
468 self._pack_int(dimid)
ValueError: '2' is not in list
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-22-4077b7ef5b8c> in <module>()
2 ds = xr.open_dataset('/home/mowglie/Downloads/rasm.nc')
3 ds.Tair.encoding['dtype'] = 'float32'
----> 4 ds.to_netcdf('/home/mowglie/rasm.nc', format='NETCDF3_CLASSIC', engine='scipy')
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/core/dataset.py in to_netcdf(self, path, mode, format, group, engine, encoding)
780 from ..backends.api import to_netcdf
781 return to_netcdf(self, path, mode, format=format, group=group,
--> 782 engine=engine, encoding=encoding)
783
784 dump = utils.function_alias(to_netcdf, 'dump')
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/api.py in to_netcdf(dataset, path, mode, format, group, engine, writer, encoding)
357 finally:
358 if sync:
--> 359 store.close()
360
361 if not sync:
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/xarray/backends/scipy_.py in close(self)
149
150 def close(self):
--> 151 self.ds.close()
152
153 def __exit__(self, type, value, tb):
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in close(self)
277 if not self.fp.closed:
278 try:
--> 279 self.flush()
280 finally:
281 self.variables = {}
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in flush(self)
387 """"""
388 if hasattr(self, 'mode') and self.mode in 'wa':
--> 389 self._write()
390 sync = flush
391
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write(self)
399 self._write_dim_array()
400 self._write_gatt_array()
--> 401 self._write_var_array()
402
403 def _write_numrecs(self):
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_array(self)
447 # Set the metadata for all variables.
448 for name in variables:
--> 449 self._write_var_metadata(name)
450 # Now that we have the metadata, we know the vsize of
451 # each record variable, so we can calculate recsize.
/home/mowglie/.bin/conda/envs/climate/lib/python3.5/site-packages/scipy/io/netcdf.py in _write_var_metadata(self, name)
465 self._pack_int(len(var.dimensions))
466 for dimname in var.dimensions:
--> 467 dimid = self._dims.index(dimname)
468 self._pack_int(dimid)
469
ValueError: '2' is not in list
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259788839,https://api.github.com/repos/pydata/xarray/issues/1106,259788839,MDEyOklzc3VlQ29tbWVudDI1OTc4ODgzOQ==,1217238,2016-11-10T19:48:04Z,2016-11-10T19:48:04Z,MEMBER,"Yes, please open a PR in xarray-data!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259788569,https://api.github.com/repos/pydata/xarray/issues/1106,259788569,MDEyOklzc3VlQ29tbWVudDI1OTc4ODU2OQ==,10050469,2016-11-10T19:46:58Z,2016-11-10T19:46:58Z,MEMBER,"Yes, with the following code the file size goes down to 9mb:
``` python
import xarray as xr
ds = xr.open_dataset('/home/tt/Downloads/rasm.nc')
ds.Tair.encoding['dtype'] = 'float32'
ds.to_netcdf('/home/tt/rasm.nc', format='NETCDF3_CLASSIC')
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259786864,https://api.github.com/repos/pydata/xarray/issues/1106,259786864,MDEyOklzc3VlQ29tbWVudDI1OTc4Njg2NA==,10050469,2016-11-10T19:40:10Z,2016-11-10T19:40:10Z,MEMBER,"I'm happy to do the conversion if you want. One could compress the file a little bit by changing the data to float32, too
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022
https://github.com/pydata/xarray/issues/1106#issuecomment-259764394,https://api.github.com/repos/pydata/xarray/issues/1106,259764394,MDEyOklzc3VlQ29tbWVudDI1OTc2NDM5NA==,1217238,2016-11-10T18:12:00Z,2016-11-10T18:12:00Z,MEMBER,"Another option would be to switch all the example files (https://github.com/pydata/xarray-data) to use netCDF3. That's probably a good idea for maximum compatibility in any case, because we can read netCDF3 with only scipy installed.
cc @jhamman
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,188565022