html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/pull/1260#issuecomment-305243114,https://api.github.com/repos/pydata/xarray/issues/1260,305243114,MDEyOklzc3VlQ29tbWVudDMwNTI0MzExNA==,1217238,2017-05-31T16:32:27Z,2017-05-31T16:32:27Z,MEMBER,"> Currently the rasterio tests are running on py36 only. Should I add rasterio to the other test suites as well? Yes, except for the one that tests bare-minimal dependencies.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-305231172,https://api.github.com/repos/pydata/xarray/issues/1260,305231172,MDEyOklzc3VlQ29tbWVudDMwNTIzMTE3Mg==,1217238,2017-05-31T15:51:24Z,2017-05-31T15:51:24Z,MEMBER,"Yes, please! On Wed, May 31, 2017 at 8:48 AM, Fabien Maussion wrote: > *@fmaussion* commented on this pull request. > ------------------------------ > > In xarray/backends/rasterio_.py > : > > > + chunks : int, tuple or dict, optional > + Chunk sizes along each dimension, e.g., ``5``, ``(5, 5)`` or > + ``{'x': 5, 'y': 5}``. If chunks is provided, it used to load the new > + DataArray into a dask array. This is an experimental feature; see the > + documentation for more details. > + cache : bool, optional > + If True, cache data loaded from the underlying datastore in memory as > + NumPy arrays when accessed to avoid reading from the underlying data- > + store multiple times. Defaults to True unless you specify the `chunks` > + argument to use dask, in which case it defaults to False. Does not > + change the behavior of coordinates corresponding to dimensions, which > + always load their data from disk into a ``pandas.Index``. > + lock : False, True or threading.Lock, optional > + If chunks is provided, this argument is passed on to > + :py:func:`dask.array.from_array`. By default, a per-variable lock is > + used when reading data from netCDF files with the netcdf4 and h5netcdf > > Now I understand much more what's going on, and also why salem was having > troubles when adding diagnostic variables to netcdf objects. Can I update > the docs all at once here? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-304481844,https://api.github.com/repos/pydata/xarray/issues/1260,304481844,MDEyOklzc3VlQ29tbWVudDMwNDQ4MTg0NA==,1217238,2017-05-27T23:23:02Z,2017-05-27T23:23:02Z,MEMBER,"I think DataArray.chunk should probably gain those parameters. There just wasn't any need for it before. On Sat, May 27, 2017 at 9:35 AM Fabien Maussion wrote: > *@fmaussion* commented on this pull request. > ------------------------------ > > In xarray/backends/rasterio_.py > : > > > + y = np.linspace(start=y0, num=ny, stop=(y0 + (ny - 1) * dy)) > + > + # Get bands > + if riods.count < 1: > + raise ValueError('Unknown dims') > + bands = np.asarray(riods.indexes) > + > + # Attributes > + attrs = {} > + if hasattr(riods, 'crs'): > + # CRS is a dict-like object specific to rasterio > + # We convert it back to a PROJ4 string using rasterio itself > + attrs['crs'] = riods.crs.to_string() > + # Maybe we'd like to parse other attributes here (for later) > + > + data = indexing.LazilyIndexedArray(RasterioArrayWrapper(riods)) > > Implementing cache went well (appart from this bug > . > > For chunking I have a question: DataArray.chunk() doesn't have > > the name_prefix, token and lock keywords. Any reason for this? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or mute > the thread > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-303153678,https://api.github.com/repos/pydata/xarray/issues/1260,303153678,MDEyOklzc3VlQ29tbWVudDMwMzE1MzY3OA==,1217238,2017-05-22T16:38:03Z,2017-05-22T16:38:03Z,MEMBER,"> side note: ""rasterio CRS objects"" are in fact strings corresponding to a PROJ4 string that will always be understood by pyproj, rasterio, gdal, etc. Examples: '+proj=aea +lat_1=-18 +lat_2=-32 +lat_0=0 +lon_0=24 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs ' or 'EPSG:4326') Ah, that seems already pretty clean then. I was confused by the repr for a `CRS()` object in your comment above: https://github.com/pydata/xarray/pull/1260#issuecomment-279064452","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-302951509,https://api.github.com/repos/pydata/xarray/issues/1260,302951509,MDEyOklzc3VlQ29tbWVudDMwMjk1MTUwOQ==,1217238,2017-05-21T17:43:45Z,2017-05-22T16:36:32Z,MEMBER,"You should be able to simply wrap RasterioArrayWrapper in an `indexing.LazilyIndexedArray` and then pass it into the xarray.DataArray constructor directly. EDIT: `open_dataset` also wrap data with `CopyOnWriteArray` and optionally `MemoryCachedArray`. These are also probably worth using for consistency, e.g., see these lines from `xarray/backends/api.py`: ```python data = indexing.CopyOnWriteArray(variable._data) if cache: data = indexing.MemoryCachedArray(data) ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-302448132,https://api.github.com/repos/pydata/xarray/issues/1260,302448132,MDEyOklzc3VlQ29tbWVudDMwMjQ0ODEzMg==,1217238,2017-05-18T15:54:49Z,2017-05-18T15:54:49Z,MEMBER,"I haven't looked at the implementation yet, but I agree that a separate function `open_rasterio` is the way to go. Structurally, this is pretty different from opening a netCDF file, which is also why I was questioning reusing the existing xarray backends system.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-279897816,https://api.github.com/repos/pydata/xarray/issues/1260,279897816,MDEyOklzc3VlQ29tbWVudDI3OTg5NzgxNg==,1217238,2017-02-15T02:13:27Z,2017-02-15T02:13:27Z,MEMBER,"> my initial idea to make them lazily evaluated might work, but in an ugly way: computing both lons and lats needs to be done in one single operation, and I'm not sure how this can be done in an elegant way This could be done sanely with dask.array ([example of multiple outputs from an element-wise function](https://github.com/dask/dask/blob/1e2bf8973ff3706e68a5a55d6392a5798bd072bd/dask/array/ufunc.py#L116-L133)). > additionally, there is no way to make them show up as coordinates (as per @shoyer 's comment: #1260 (comment)) We could make this work, it just looks pretty hacky.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-279101252,https://api.github.com/repos/pydata/xarray/issues/1260,279101252,MDEyOklzc3VlQ29tbWVudDI3OTEwMTI1Mg==,1217238,2017-02-11T00:14:36Z,2017-02-11T00:14:36Z,MEMBER,"Getting optional coordinates is hard is because the current ""backends"" system in xarray was really only designed for handling netCDF-like libraries. For example, every backend gets passed through `xarray.conventions.decode_cf` to use CF conventions to decode its data. You would actually need to add a ""global attribute"" specifying `coords='lat lon'` to set coordinates variables with this system. One alternative would be to simply write a function for reading files with rasterio straight into xarray Dataset objects. There are probably hacks that can get this working with data stores but this really calls out for the bigger refactor like https://github.com/pydata/xarray/pull/1087. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158 https://github.com/pydata/xarray/pull/1260#issuecomment-279070566,https://api.github.com/repos/pydata/xarray/issues/1260,279070566,MDEyOklzc3VlQ29tbWVudDI3OTA3MDU2Ng==,1217238,2017-02-10T21:28:02Z,2017-02-10T21:28:02Z,MEMBER,"> The order of the variables and coordinates is random. Most likely you're using a dict instead of an OrderedDict somewhere. > Is it possible to make the x and y coords lazy? See here for an example of lazily computing arrays: https://github.com/pydata/xarray/blob/62333208fc2a80c05848a12de67a10f00a6610a1/xarray/conventions.py#L314 There are other examples in that file, for decoding CF conventions. > I'd like to have the lon and lat variables listed as optional coordinates Can you clarify what you mean by an optional coordinate? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,206905158