html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/1260#issuecomment-305417871,https://api.github.com/repos/pydata/xarray/issues/1260,305417871,MDEyOklzc3VlQ29tbWVudDMwNTQxNzg3MQ==,1392657,2017-06-01T07:53:59Z,2017-06-01T07:53:59Z,CONTRIBUTOR,"Hey @fmaussion, is this ready for me to try out again? I wanted to let you and @shoyer iterate first.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1382#issuecomment-303955629,https://api.github.com/repos/pydata/xarray/issues/1382,303955629,MDEyOklzc3VlQ29tbWVudDMwMzk1NTYyOQ==,1392657,2017-05-25T08:36:48Z,2017-05-25T08:36:48Z,CONTRIBUTOR,"Very welcome, @shoyer! On Thu, May 25, 2017 at 4:06 AM, Stephan Hoyer wrote: > Thanks @gidden ! > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,223623733 https://github.com/pydata/xarray/pull/1423#issuecomment-303657265,https://api.github.com/repos/pydata/xarray/issues/1423,303657265,MDEyOklzc3VlQ29tbWVudDMwMzY1NzI2NQ==,1392657,2017-05-24T08:37:57Z,2017-05-24T08:37:57Z,CONTRIBUTOR,"TIL, thanks! On Wed, May 24, 2017 at 10:34 AM, Stephan Hoyer wrote: > *@shoyer* commented on this pull request. > ------------------------------ > > In xarray/tests/__init__.py > : > > > yield > - assert len(w) > 0 > - assert any(message in str(wi.message) for wi in w) > + assert len(w) > 0 > > Yes, this was intentional. Python with blocks don't define scopes for > variables, so w is still defined outside the block. In this case, the with > block only indicates where warnings should be caught. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,230937851 https://github.com/pydata/xarray/pull/1382#issuecomment-303653296,https://api.github.com/repos/pydata/xarray/issues/1382,303653296,MDEyOklzc3VlQ29tbWVudDMwMzY1MzI5Ng==,1392657,2017-05-24T08:21:25Z,2017-05-24T08:21:25Z,CONTRIBUTOR,"sounds good! On Wed, May 24, 2017 at 10:01 AM, Stephan Hoyer wrote: > @gidden this isn't you -- pytest broke us > (I'll merge #1423 with a fix > shortly) > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,223623733 https://github.com/pydata/xarray/pull/1382#issuecomment-303642200,https://api.github.com/repos/pydata/xarray/issues/1382,303642200,MDEyOklzc3VlQ29tbWVudDMwMzY0MjIwMA==,1392657,2017-05-24T07:32:25Z,2017-05-24T07:32:25Z,CONTRIBUTOR,"hey @shoyer, this is failing one each of a dataset and dataarray test. I am not sure why, could you please assist? thanks!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,223623733 https://github.com/pydata/xarray/pull/1260#issuecomment-303370889,https://api.github.com/repos/pydata/xarray/issues/1260,303370889,MDEyOklzc3VlQ29tbWVudDMwMzM3MDg4OQ==,1392657,2017-05-23T11:34:47Z,2017-05-23T11:34:47Z,CONTRIBUTOR,Sounds good!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-303330504,https://api.github.com/repos/pydata/xarray/issues/1260,303330504,MDEyOklzc3VlQ29tbWVudDMwMzMzMDUwNA==,1392657,2017-05-23T08:37:59Z,2017-05-23T08:37:59Z,CONTRIBUTOR,"Note that the above `tiff` was made with something like the following code: ``` res = 0.5 nlat = 180 nlon = 360 left_lon = -180 upper_lat = 90 crs = 'epsg:4326' profile = { 'affine': rio.Affine(res, 0.0, left_lon, 0.0, -res, upper_lat), 'dtype': 'int32', 'height': int(nlat / res), 'width': int(nlon / res), 'nodata': '-1', 'crs': crs, } ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-303330186,https://api.github.com/repos/pydata/xarray/issues/1260,303330186,MDEyOklzc3VlQ29tbWVudDMwMzMzMDE4Ng==,1392657,2017-05-23T08:36:46Z,2017-05-23T08:36:46Z,CONTRIBUTOR,"Hey @fmaussion, I guess my question here is as follows: if a raster was generated by rasterio (thus using a dictionary representation of the CRS object even if a string is provided), should xarrary then fail to write to netcdf with that dataset? If so, then that means that all users will have to do ``` ds = open_rasterio('file.tiff') ds.attrs['crs'] = ds.attrs['crs'].to_string() ds.to_netcdf('file.nc') ``` Would it be better to use the string representation of the CRS internally after reading in? I think this would solve @shoyer's concern about onboarding non-primitive datatypes. As an aside, even my above suggestion did not work.. I'm not sure why. ``` In [33]: ds = xr.open_rasterio('./isimip_centroid_0_5.tiff') In [34]: ds Out[34]: Dimensions: (band: 1, x: 720, y: 360) Coordinates: * y (y) float64 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 ... * x (x) float64 -180.0 -179.5 -179.0 -178.5 -178.0 -177.5 -177.0 ... * band (band) int64 1 Data variables: raster (band, y, x) int32 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ... Attributes: crs: CRS({'init': u'epsg:4326'}) In [35]: ds.attrs['crs'] = ds.attrs['crs'].to_string() In [36]: ds Out[36]: Dimensions: (band: 1, x: 720, y: 360) Coordinates: * y (y) float64 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 ... * x (x) float64 -180.0 -179.5 -179.0 -178.5 -178.0 -177.5 -177.0 ... * band (band) int64 1 Data variables: raster (band, y, x) int32 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ... Attributes: crs: +init=epsg:4326 In [37]: ds.to_netcdf('test.nc') --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () ----> 1 ds.to_netcdf('test.nc') /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/core/dataset.pyc in to_netcdf(self, path, mode, format, group, engine, encoding, unlimited_dims) 975 return to_netcdf(self, path, mode, format=format, group=group, 976 engine=engine, encoding=encoding, --> 977 unlimited_dims=unlimited_dims) 978 979 def __unicode__(self): /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in to_netcdf(dataset, path_or_file, mode, format, group, engine, writer, encoding, unlimited_dims) 588 # validate Dataset keys, DataArray names, and attr keys/values 589 _validate_dataset_names(dataset) --> 590 _validate_attrs(dataset) 591 592 try: /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in _validate_attrs(dataset) 119 for variable in dataset.variables.values(): 120 for k, v in variable.attrs.items(): --> 121 check_attr(k, v) 122 123 /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in check_attr(name, value) 110 'string, ndarray or a list/tuple of ' 111 'numbers/strings for serialization to netCDF ' --> 112 'files'.format(value)) 113 114 # Check attrs on the dataset itself TypeError: Invalid value for attr: CRS({'init': u'epsg:4326'}) must be a number string, ndarray or a list/tuple of numbers/strings for serialization to netCDF files ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-303317475,https://api.github.com/repos/pydata/xarray/issues/1260,303317475,MDEyOklzc3VlQ29tbWVudDMwMzMxNzQ3NQ==,1392657,2017-05-23T07:47:35Z,2017-05-23T07:57:34Z,CONTRIBUTOR,"Hey @fmaussion, should we also do any attribute checking, or should we let the user fails as follows? I recently was testing this out and made a `*.tiff` with a shapefile as read from `fiona`. I got the following error: ``` In [12]: ds Out[12]: Dimensions: (band: 1, x: 720, y: 360) Coordinates: * y (y) float64 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 ... * x (x) float64 -180.0 -179.5 -179.0 -178.5 -178.0 -177.5 -177.0 ... * band (band) int64 1 Data variables: raster (band, y, x) int32 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ... Attributes: crs: CRS({'init': u'epsg:4326'}) In [13]: ds.to_netcdf('test.nc') --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () ----> 1 ds.to_netcdf('test.nc') /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/core/dataset.pyc in to_netcdf(self, path, mode, format, group, engine, encoding, unlimited_dims) 975 return to_netcdf(self, path, mode, format=format, group=group, 976 engine=engine, encoding=encoding, --> 977 unlimited_dims=unlimited_dims) 978 979 def __unicode__(self): /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in to_netcdf(dataset, path_or_file, mode, format, group, engine, writer, encoding, unlimited_dims) 588 # validate Dataset keys, DataArray names, and attr keys/values 589 _validate_dataset_names(dataset) --> 590 _validate_attrs(dataset) 591 592 try: /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in _validate_attrs(dataset) 114 # Check attrs on the dataset itself 115 for k, v in dataset.attrs.items(): --> 116 check_attr(k, v) 117 118 # Check attrs on each variable within the dataset /home/gidden/.local/lib/python2.7/site-packages/xarray-0.9.5_34_g48c7268-py2.7.egg/xarray/backends/api.pyc in check_attr(name, value) 110 'string, ndarray or a list/tuple of ' 111 'numbers/strings for serialization to netCDF ' --> 112 'files'.format(value)) 113 114 # Check attrs on the dataset itself TypeError: Invalid value for attr: CRS({'init': u'epsg:4326'}) must be a number string, ndarray or a list/tuple of numbers/strings for serialization to netCDF files ``` And it seems like this happens whether I use a dictionary originally or just a string. Perhaps this is a slightly larger issue with cleaning the CRS object such that it is netcdf compatible? Does this mean just a ``` crs = ' '.join('+{}={}'.format(k, v) for k, v in crs.items()) ``` '?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1070#issuecomment-303033917,https://api.github.com/repos/pydata/xarray/issues/1070,303033917,MDEyOklzc3VlQ29tbWVudDMwMzAzMzkxNw==,1392657,2017-05-22T08:28:53Z,2017-05-22T08:28:53Z,CONTRIBUTOR,"@byersiiasa indeed I am, in fact I'm reviewing #1260 which overtook this one =)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186326698 https://github.com/pydata/xarray/pull/1260#issuecomment-302945676,https://api.github.com/repos/pydata/xarray/issues/1260,302945676,MDEyOklzc3VlQ29tbWVudDMwMjk0NTY3Ng==,1392657,2017-05-21T16:02:01Z,2017-05-21T16:02:01Z,CONTRIBUTOR,"Hey @fmaussion, I personally think some lat/lon support should be included here. I would lean toward *1* personally, but would be interested in what others think.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1382#issuecomment-302944802,https://api.github.com/repos/pydata/xarray/issues/1382,302944802,MDEyOklzc3VlQ29tbWVudDMwMjk0NDgwMg==,1392657,2017-05-21T15:46:47Z,2017-05-21T15:46:47Z,CONTRIBUTOR,"ok, should be cleaned up and ready to go @shoyer ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,223623733 https://github.com/pydata/xarray/pull/1393#issuecomment-302943350,https://api.github.com/repos/pydata/xarray/issues/1393,302943350,MDEyOklzc3VlQ29tbWVudDMwMjk0MzM1MA==,1392657,2017-05-21T15:22:07Z,2017-05-21T15:22:07Z,CONTRIBUTOR,"hey @shoyer, @fmaussion. test decorator added with tutorial dataset unit test. i think this is good to go.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435 https://github.com/pydata/xarray/pull/1393#issuecomment-298969483,https://api.github.com/repos/pydata/xarray/issues/1393,298969483,MDEyOklzc3VlQ29tbWVudDI5ODk2OTQ4Mw==,1392657,2017-05-03T16:50:17Z,2017-05-03T16:50:17Z,CONTRIBUTOR,"Hey @shoyer, if I have time later I will try to address the unit test issue. If I'm too late, feel free to pull this in the meantime. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435 https://github.com/pydata/xarray/pull/1393#issuecomment-298942200,https://api.github.com/repos/pydata/xarray/issues/1393,298942200,MDEyOklzc3VlQ29tbWVudDI5ODk0MjIwMA==,1392657,2017-05-03T15:15:15Z,2017-05-03T15:15:15Z,CONTRIBUTOR,"ok, this should be good to go","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435 https://github.com/pydata/xarray/pull/1393#issuecomment-298755199,https://api.github.com/repos/pydata/xarray/issues/1393,298755199,MDEyOklzc3VlQ29tbWVudDI5ODc1NTE5OQ==,1392657,2017-05-02T20:43:18Z,2017-05-02T20:43:18Z,CONTRIBUTOR,"Ok, sounds good. `shutil` was no longer needed, but the md5 checksum was implemented as private. So I think all is well. On Tue, May 2, 2017 at 2:32 PM, Stephan Hoyer wrote: > Thanks for taking this on @gidden . > > Also not sure how best to import shutil. I followed the current pattern > for os, but am not sure why that pattern exists in this file. > > I think this exists to work around the fact that IPython shows everything > that isn't private in auto-complete (ipython/ipykernel#129 > ). But we should > probably switch back to normal imports in this module -- this is really > more of an IPython issue. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435 https://github.com/pydata/xarray/pull/1393#issuecomment-298714845,https://api.github.com/repos/pydata/xarray/issues/1393,298714845,MDEyOklzc3VlQ29tbWVudDI5ODcxNDg0NQ==,1392657,2017-05-02T18:07:57Z,2017-05-02T18:07:57Z,CONTRIBUTOR,"Successful locally with pydata/xarray-data#9: ``` In [3]: xr.tutorial.load_dataset('air_temperature', github_url='https://github.com/gidden/xarray-data', branch='md5') Out[3]: Dimensions: (lat: 25, lon: 53, time: 2920) Coordinates: * lat (lat) float32 75.0 72.5 70.0 67.5 65.0 62.5 60.0 57.5 55.0 52.5 ... * lon (lon) float32 200.0 202.5 205.0 207.5 210.0 212.5 215.0 217.5 ... * time (time) datetime64[ns] 2013-01-01 2013-01-01T06:00:00 ... Data variables: air (time, lat, lon) float64 241.2 242.5 243.5 244.0 244.1 243.9 ... Attributes: Conventions: COARDS title: 4x daily NMC reanalysis (1948) description: Data is from NMC initialized reanalysis\n(4x/day). These a... platform: Model references: http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly... ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435 https://github.com/pydata/xarray/pull/1393#issuecomment-298705664,https://api.github.com/repos/pydata/xarray/issues/1393,298705664,MDEyOklzc3VlQ29tbWVudDI5ODcwNTY2NA==,1392657,2017-05-02T17:34:06Z,2017-05-02T17:34:06Z,CONTRIBUTOR,"It seems like urlretrieve is coming directly from urllib. It seems in both Python 2 and 3 ContentTooShort error should be thrown and isnt. Perhaps checksums are the right way to go. On Tue, May 2, 2017 at 7:14 PM, Fabien Maussion wrote: > *@fmaussion* commented on this pull request. > ------------------------------ > > In xarray/tutorial.py > : > > > > url = '/'.join((github_url, 'raw', 'master', fullname)) > - _urlretrieve(url, localfile) > + _urlretrieve(url, tmpfile) > + > + if not _os.path.exists(tmpfile): > + raise ValueError('File could not be downloaded, please try again') > + > + _shutil.move(tmpfile, localfile) > > I thought that the issue wasn't that the file isn't downloaded, but rather > that it is incompletely downloaded? Will this new temporary step than solve > this? > > Is there a way to make urlretrieve more robust, with checksums or > something? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > , > or mute the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,225672435