issue_comments

@ajelenak-thg thanks for your patience here. I need to think a little bit harder about what the right API for controlling the default engine is, but this work is definitely appreciated (and will probably make it in for the next release).

Now that the tests are passing again, is there anything else left to change?

I've refactored setting the I/O engine option as per our discussion. Hopefully, it captures now all the requested functionality.

One more minor complexity is that there are two version of netCDF, not all of which are supported by all tools:

• netcdf4 support netcdf3 and netcdf4
• scipy only supports netcdf3
• h5netcdf only supports netcdf4.
• pynio supports both netcdf3 and netcdf4.

So will need to distinguish between URLs (opendap) and these two types of netCDF files. To do this, we'll need to look at the first few bytes of each file: - netCDF4/HDF5 starts with the 8-byte signature b'\x89HDF\r\n\x1a\n' - netCDF4 starts with b'CDF\x00' or b'CDF\x01'

Let's see if we can get this PR over the line... 😄

A list of engines would need some way of declaring their I/O capabilities: only file-based, only HTTP-based, or both. Something like:

```python io_engines = [ {'engine': 'netcdf4', 'capabilities': ['file', 'http']},

{'engine': 'pydap', 
 'capabilities': ['http']},

{'engine': 'scipy', 
 'capabilities': ['file']},

{'engine': 'h5netcdf',
 'capabilities': ['file']},

] ```

On xarray import or any time this option would change, the list of engines would be checked to remove unavailable engines.

Maybe it would make more sense to set a list of engines to try in order? e.g., io_engines=['pydap', 'h5netcdf', 'scipy'] could use pydap for web requests, h5netcdf for netCDF4 and scipy for netCDF3. The default would be something like ['netcdf4', 'pydap', 'scipy', 'h5netcdf']

Yes, there could be more I/O engine options. How about file_engine and web_engine? Keeping the naming more generic as there may be other file-based formats or data web services in the future.

On the other hand, setting this global option should indicate a willingness to accept the consequences. If automatic selection of the optional I/O engine is preferred, this global option should not be set.

We should probably have separate options for opendap/netcdf. It would not be so useful to try to use pydap to read a netcdf file.

How about io_engine for the option's name? The data can come from an OPeNDAP server as well.

I have reverted to the original engine=None in the functions/methods and use python engine = engine or OPTIONS['io_engine'] to assign the correct engine value.

I like the direction this is going in, but as @MaximilianR notes it won't work to set this in default function arguments.

Also, I would probably pick a more descriptive name for the option, e.g., netcdf_engine.

This looks great so far!

But have you tested this? I have a suspicion that putting OPTIONS['engine'] as a default arg won't work: whenever the relevant function is imported, the default will be set to the value at that time, and won't change if the options dictionary is later modified.

You could have a default arg of None and then look up the option within the function, if my suspicions are right