issue_comments

thanks @scottyhq !

I think we're good here. I made one minor tweak to the windows fix @scottyhq implemented. I plan to merge this on Monday if I don't hear any objections.

Hello @scottyhq! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2019-03-15 23:35:28 UTC

@scottyhq - can you add note to the what's new page?

From what I can tell, I don't think the failing tests are related to this PR.

thanks for the input @shoyer, I attempted to tidy up a bit and in the process re-ordered some things such as adding an 'engine' check at the top of open_dataset(). backend tests are passing locally on my machine. hopefully i didn't add too much here or overstep!

I don't think it's essential to have an integration test doing real network access in xarray, so I would consider just dropping that part instead.

On Tue, Mar 5, 2019 at 6:10 PM Ryan Abernathey notifications@github.com wrote:

@rabernat commented on this pull request.

In xarray/tests/test_backends.py https://github.com/pydata/xarray/pull/2782#discussion_r262767473:

@@ -1955,6 +1955,39 @@ def test_dump_encodings_h5py(self): assert actual.x.encoding['compression_opts'] is None

+@requires_h5fileobj +class TestH5NetCDFFileObject(TestH5NetCDFData): + h5py = pytest.importorskip('h5py', minversion='2.9.0') + engine = 'h5netcdf' + + @network + def test_h5remote(self): + # alternative: http://era5-pds.s3.amazonaws.com/2008/01/main.nc

• import requests
• url = ('https://www.unidata.ucar.edu/'
• 'software/netcdf/examples/test_hgroups.nc')

Rather than going over the network, it might be quite easy to fire up a http.server.SimpleHTTPRequestHandler https://docs.python.org/3/library/http.server.html#http.server.SimpleHTTPRequestHandler as part of a fixture. This would allow us to test the remote capability without internet (and without depending on a third party to host a file.)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/2782#pullrequestreview-211014423, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1puGwIDrvG3NGvfq5oXqwdtbHSIDks5vTyOkgaJpZM4bGMFq .

@shoyer , it would be great to have your feedback on these recent changes now that h5netcdf 0.7 is out. There's a bit more logic required in api.py now that scipy isn't the only backend that is able to read file-like objects (and people may not specify engine= when opening datasets)

test_backends.py passes locally for me except for TestValidateAttrs.test_validating_attrs... not sure why.

Also, per your comment here: https://github.com/shoyer/h5netcdf/pull/51#issuecomment-467591446, I think it would be great to get a few small netcdf4/hdf test files in https://github.com/pydata/xarray-data.

This looks great!

I'll note one minor extension: you could look at the first few bytes of the file (the "magic number") to determine if it's a netCDF3 or netCDF4 file, and hence whether it can be opened with scipy or h5netcdf:

• CDF\001 or CDF\002 would indicate netCDF3 (use scipy)
• \211HDF\r\n\032\n would indicate netCDF4/HDF5 (use h5netcdf)

I'm glad to see this. I'll also be curious to see what the performance will look like.

cc @llllllllll