issue_comments

The only way we could make reading a gzipped netCDF4 file is to load the entire file into memory. That's why we didn't support this before. It's also less relevant for netCDF4, because netCDF4 supports in-file compression directly.

With netCDF3, we can use scipy's netcdf reader, which supports Python file objects. But netCDF4-Python does not support Python file objects.

This issue is concerned about supporting paths ending with .gz in remote URLs, which are not local files but rather files exposed via the OpenDAP protocol over a network. If the DAP server can server a file with a .gz extension then xarray should be OK with it, too.

Currently the way we handle this is that the only test that accessing remote resources is the pydap test, which only runs in one build (not required to pass).

I am reluctant to merge this without having any way to test the logic. Without automated tests this issue is likely to recur.

That said, I suppose we could leave the refactoring for a TODO. Let's add a note on that and also one minimal test to verify that we raise an error if you try to use engine='netcdf4' with a path to local gzipped file.

We do have existing tests for backends: https://github.com/pydata/xarray/blob/master/xarray/test/test_backends.py

This includes a test accessing an OpenDAP dataset from the OpenDAP test server (via pydap, at the end). But in my experience, there servers are somewhat unreliable (maybe available 90%), so we don't require that test to pass for the build to pass. Also, even in the best case scenario network access is slow. So it would be nice to modularize this enough that our logic is testable without actually using opendap.

I think this is probably correct, but the heuristics here are starting to get convoluted enough that I worry about test coverage. Is there any way we can test this? Maybe try to pull the gzip logic into a helper function (an extended variant of _get_default_engine) that we could test?