issues

• Allows conversion to and from: NetCDF4, scipy.io.netcdf and in memory storage.
• Added general test cases, and cases for specific backend stores.

Previously conflicting _FillValue and missing_value only raised an AssertionError, now it's more informative.

The right solution to this is to figure out how to successfully round trip endian-ness, but that seems to be a deeper issue inside netCDF4 (https://github.com/Unidata/netcdf4-python/issues/346)

Instead we force all data to little endian before netCDF4 write.

This makes it possible to read datasets that don't follow CF datetime conventions perfectly, such as the following example which (surprisingly) comes from NCEP/NCAR (you'd think they would follow CF!)

``` ds = xray.open_dataset('http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GEFS/Global_1p0deg_Ensemble/members/GEFS_Global_1p0deg_Ensemble_20150114_1200.grib2/GC') print ds['time'].encoding['units']

u'Hour since 2015-01-14T12:00:00Z' ```

Restructured Backends to make CF conventions handling consistent.

Among other things this includes: - EncodedDataStores which can wrap other stores and allow for modular encoding/decoding. - Trivial indices ds['x'] = ('x', np.arange(10)) are no longer stored on disk and are only created when accessed. - AbstractDataStore API change. Shouldn't effect external users. - missing_value attributes now function like _FillValue

All current tests are passing (though it could use more new ones).

Removed the object oriented encoding/decoding scheme in favor of a model where encoding/decoding happens when a dataset is stored to/ loaded from a DataStore.

Conventions can now be enforced at the DataStore level by overwriting the Datastore.store() and Datastore.load() methods, or as an optional arg to Dataset.load_store, Dataset.dump_to_store.

Includes miscellaneous cleanup.

``` Data objects now have a swap-able backend store.

• Allows conversion to and from: NetCDF4, scipy.io.netcdf and in memory storage.
• Added general test cases, and cases for specific backend stores.
• Dataset.translate() can now optionally copy the object.
• Fixed most unit tests, test_translate_consistency still fails. ```

decode_cf_variable was still using da.data instead of da.values.

It now also works with DataArray as input.

same as #153, but without tests.

concat_over=True indicates that concat should concat over all variables that are not the same in the set of datasets that are to be concatenated.

Internally Datasets convert time coordinates to pandas.DatetimeIndex. The backend function convert_to_cf_variable will convert these datetimes back to CF style times, but the original units were not being preserved.

Fixed a bug in which writes would fail if Datasets contained 0d variables.

Also added the ability to open Datasets directly from NetCDF3 bytestrings.

Small bug fix, and a test.

This was causing an error in the following situation:

ds = xray.Dataset() ds['time'] = ('time', [np.datetime64('2001-05-01') for i in range(5)]) ds['variable'] = ('time', np.arange(5.)) ds.to_netcdf('test.nc') ds = xray.open_dataset('./test.nc') ss = ds.indexed(time=slice(0, 2)) ss.dumps()

Thanks @shoyer for the fix.

In an attempt to coerce all datetime arrays to nano second resolutoin utils.as_safe_array() was creating copies of any datetime64 array (via the astype method). This was causing unexpected behavior (bugs) for things such as concatenation over times. (see below).

``` import xray import pandas as pd

ds = xray.Dataset() ds['time'] = ('time', pd.date_range('2011-09-01', '2011-09-11')) times = [ds.indexed(time=[i]) for i in range(10)] ret = xray.Dataset.concat(times, 'time') print ret['time']

<xray.DataArray 'time' (time: 10)> array(['1970-01-02T07:04:40.718526408-0800', '1969-12-31T16:00:00.099966608-0800', '1969-12-31T16:00:00.041748384-0800', '1969-12-31T16:00:00.041748360-0800', '1969-12-31T16:00:00.041748336-0800', '1969-12-31T16:00:00.041748312-0800', '1969-12-31T16:00:00.041748288-0800', '1969-12-31T16:00:00.041748264-0800', '1969-12-31T16:00:00.041748240-0800', '1969-12-31T16:00:00.041748216-0800'], dtype='datetime64[ns]') Attributes: Empty ```

decode_cf_variable will not save encoding for any 'object' dtypes.

When encoding cf variables check if dtype is np.datetime64 as well as DatetimeIndex.

fixes akleeman/xray/issues/39

BUG: dataset_repr was failing on empty datasets.