html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/1075#issuecomment-357130204,https://api.github.com/repos/pydata/xarray/issues/1075,357130204,MDEyOklzc3VlQ29tbWVudDM1NzEzMDIwNA==,1217238,2018-01-12T03:03:22Z,2018-01-12T03:03:22Z,MEMBER,"We could potentially add a from_memory() constructor to NetCDF4DataStore to
simplify this process.
On Thu, Jan 11, 2018 at 6:27 PM Michael Delgado
wrote:
> yes! Thanks @jhamman and @shoyer
> . I hadn't tried it yet, but just did. worked
> great!
>
> In [1]: import xarray as xr
> ...: import requests
> ...: import netCDF4
> ...:
> ...: %matplotlib inline
>
> In [2]: res = requests.get(
> ...: 'http://nasanex.s3.amazonaws.com/NEX-GDDP/BCSD/rcp45/day/atmos/tasmin/' +
> ...: 'r1i1p1/v1.0/tasmin_day_BCSD_rcp45_r1i1p1_CESM1-BGC_2073.nc')
>
> In [3]: res.status_code
> Out [3]: 200
>
> In [4]: res.headers['content-type']
> Out [4]: 'application/x-netcdf'
>
> In [5]: nc4_ds = netCDF4.Dataset('tasmin_day_BCSD_rcp45_r1i1p1_CESM1-BGC_2073', memory=res.content)
>
> In [6]: store = xr.backends.NetCDF4DataStore(nc4_ds)
>
> In [7]: ds = xr.open_dataset(store)
>
> In [8]: ds.tasmin.isel(time=0).plot()
> /global/home/users/mdelgado/git/public/xarray/xarray/plot/utils.py:51: FutureWarning: 'pandas.tseries.converter.register' has been moved and renamed to 'pandas.plotting.register_matplotlib_converters'.
> converter.register()
> Out [8]:
>
> [image: output_7_2]
>
>
> In [9]: ds
> Out [9]:
>
> Dimensions: (lat: 720, lon: 1440, time: 365)
> Coordinates:
> * time (time) datetime64[ns] 2073-01-01T12:00:00 2073-01-02T12:00:00 ...
> * lat (lat) float32 -89.875 -89.625 -89.375 -89.125 -88.875 -88.625 ...
> * lon (lon) float32 0.125 0.375 0.625 0.875 1.125 1.375 1.625 1.875 ...
> Data variables:
> tasmin (time, lat, lon) float64 ...
> Attributes:
> parent_experiment: historical
> parent_experiment_id: historical
> parent_experiment_rip: r1i1p1
> Conventions: CF-1.4
> institution: NASA Earth Exchange, NASA Ames Research C...
> institute_id: NASA-Ames
> realm: atmos
> modeling_realm: atmos
> version: 1.0
> downscalingModel: BCSD
> experiment_id: rcp45
> frequency: day
> realization: 1
> initialization_method: 1
> physics_version: 1
> tracking_id: 1865ff49-b20c-4268-852a-a9503efec72c
> driving_data_tracking_ids: N/A
> driving_model_ensemble_member: r1i1p1
> driving_experiment_name: historical
> driving_experiment: historical
> model_id: BCSD
> references: BCSD method: Thrasher et al., 2012, Hydro...
> DOI: http://dx.doi.org/10.7292/W0MW2F2G
> experiment: RCP4.5
> title: CESM1-BGC global downscaled NEX CMIP5 Cli...
> contact: Dr. Rama Nemani: rama.nemani@nasa.gov, Dr...
> disclaimer: This data is considered provisional and s...
> resolution_id: 0.25 degree
> project_id: NEXGDDP
> table_id: Table day (12 November 2010)
> source: BCSD 2014
> creation_date: 2015-01-07T19:18:31Z
> forcing: N/A
> product: output
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
>
> .
>
","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-357115879,https://api.github.com/repos/pydata/xarray/issues/1075,357115879,MDEyOklzc3VlQ29tbWVudDM1NzExNTg3OQ==,2443309,2018-01-12T01:26:09Z,2018-01-12T01:26:09Z,MEMBER,"@delgadom - did you find a solution here?
A few more references, we're exploring ways to do this in the Pangeo project using Fuse (https://github.com/pangeo-data/pangeo/issues/52). There is a s3 equivalent of the gcsfs library used in that issue: https://github.com/dask/s3fs","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-347707575,https://api.github.com/repos/pydata/xarray/issues/1075,347707575,MDEyOklzc3VlQ29tbWVudDM0NzcwNzU3NQ==,1217238,2017-11-29T00:09:09Z,2017-11-29T00:09:31Z,MEMBER,"@delgadom Yes, that should work (I haven't tested it, but yes in principle it should all work now).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-346214120,https://api.github.com/repos/pydata/xarray/issues/1075,346214120,MDEyOklzc3VlQ29tbWVudDM0NjIxNDEyMA==,1217238,2017-11-22T01:23:42Z,2017-11-22T01:23:42Z,MEMBER,"Just to clarify: I wrote about that we use could support initializing a Dataset from a netCDF4 file image. But this wouldn't help yet for streaming access.
Initializing a Dataset from a netCDF4 file image should actually work with the latest versions of xarray and netCDF4-python:
```python
nc4_ds = netCDF4.Dataset('arbitrary-name', memory=netcdf_bytes)
store = xarray.backends.NetCDF4DataStore(nc4_ds)
ds = xarray.open_dataset(store)
```","{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-307297489,https://api.github.com/repos/pydata/xarray/issues/1075,307297489,MDEyOklzc3VlQ29tbWVudDMwNzI5NzQ4OQ==,1217238,2017-06-09T05:16:40Z,2017-06-09T05:16:40Z,MEMBER,"Yes, we could support initializing a Dataset from netCDF4 file image in a `bytes` object.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-307277552,https://api.github.com/repos/pydata/xarray/issues/1075,307277552,MDEyOklzc3VlQ29tbWVudDMwNzI3NzU1Mg==,1197350,2017-06-09T02:21:55Z,2017-06-09T02:21:55Z,MEMBER,Is this issue resolvable now that unidata/netcdf4-python#652 has been merged?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655
https://github.com/pydata/xarray/issues/1075#issuecomment-258018067,https://api.github.com/repos/pydata/xarray/issues/1075,258018067,MDEyOklzc3VlQ29tbWVudDI1ODAxODA2Nw==,1217238,2016-11-02T22:24:43Z,2016-11-02T22:25:08Z,MEMBER,"This _does_ work for netCDF3 files, if you provide a file-like object (e.g., wrapped in `BytesIO`) or set `engine='scipy'`.
Unfortunately, this is a netCDF4/HDF5 file:
```
>>> data.raw.read(8)
'\x89HDF\r\n\x1a\n'
```
And as yet, there is no support for reading from file-like objects in either h5py (https://github.com/h5py/h5py/issues/552) or python-netCDF4 (https://github.com/Unidata/netcdf4-python/issues/295). So we're currently stuck :(.
One possibility is to use the new HDF5 library [pyfive](https://github.com/jjhelmus/pyfive) with h5netcdf (https://github.com/shoyer/h5netcdf/issues/25). But pyfive doesn't have enough features yet to read netCDF files.
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,186895655