issues

What happened?

In testing an upgrade to the latest version of xarray in our systems, I noticed this test failing: ``` def test_weighted_operations_nonequal_coords(): # There are no weights for a == 4, so that data point is ignored. weights = DataArray(np.random.randn(4), dims=("a",), coords=dict(a=[0, 1, 2, 3])) data = DataArray(np.random.randn(4), dims=("a",), coords=dict(a=[1, 2, 3, 4])) check_weighted_operations(data, weights, dim="a", skipna=None)

    q = 0.5
    result = data.weighted(weights).quantile(q, dim="a")
    # Expected value computed using code from [https://aakinshin.net/posts/weighted-quantiles/](https://www.google.com/url?q=https://aakinshin.net/posts/weighted-quantiles/&sa=D) with values at a=1,2,3
    expected = DataArray([0.9308707], coords={"quantile": [q]}).squeeze()

  assert_allclose(result, expected)

E AssertionError: Left and right DataArray objects are not close E
E Differing values: E L E array(0.919569) E R E array(0.930871) ```

It appears that this test is hard-coded to match a particular random number seed, which in turn would fix the resutls of np.random.randn().

What did you expect to happen?

Whenever possible, Xarray's own tests should avoid relying on particular random number generators, e.g., in this case we could specify random numbers instead.

A back-up option would be to explicitly set random seed locally inside the tests, e.g., by creating a np.random.RandomState() with a fixed seed and using that. The global random state used by np.random.randn() is sensitive to implementation details like the order in which tests are run.

Minimal Complete Verifiable Example

No response

Relevant log output

No response

Anything else we need to know?

No response

Environment

...

Exporting and xarray object with a string dimension which only consists of a single value fails using python 3 and xarray 0.8.2.

Here is an example to reproduce the error using the toy weather data from the tutorials:

``` python import xarray as xr import numpy as np import pandas as pd

times = pd.date_range('2000-01-01', '2001-12-31', name='time') annual_cycle = np.sin(2 * np.pi * (times.dayofyear / 365.25 - 0.28))

base = 10 + 15 * annual_cycle.reshape(-1, 1) tmin_values = base + 3 * np.random.randn(annual_cycle.size, 3) tmax_values = base + 10 + 3 * np.random.randn(annual_cycle.size, 3)

ds = xr.Dataset({'tmin': (('time', 'location'), tmin_values), 'tmax': (('time', 'location'), tmax_values)}, {'time': times, 'location': ['IA', 'IN', 'IL']}) ```

The following function call fails for me with a TypeError: len() of unsized object:

python ds.sel(location="IA").to_netcdf("asdf", engine="netcdf4")

The lower part of the traceback:

``` pytb D:\Anaconda3\lib\site-packages\xarray\backends\netCDF4_.py in prepare_variable(self, name, variable, check_encoding) 250 251 if self.format == 'NETCDF4': --> 252 variable, datatype = _nc4_values_and_dtype(variable) 253 else: 254 variable = encode_nc3_variable(variable)

D:\Anaconda3\lib\site-packages\xarray\backends\netCDF4_.py in _nc4_values_and_dtype(var) 75 if var.dtype.kind == 'U': 76 # this entire clause should not be necessary with netCDF4>=1.0.9 ---> 77 if len(var) > 0: 78 var = var.astype('O') 79 dtype = str

D:\Anaconda3\lib\site-packages\xarray\core\utils.py in len(self) 371 return self.shape[0] 372 except IndexError: --> 373 raise TypeError('len() of unsized object') 374 375 ```

The export works fine both without the selection:

python ds.to_netcdf("asdf", engine="netcdf4")

And with dropping the dimension after:

python ds.sel(location="IA").drop("location").to_netcdf("asdf", engine="netcdf4")

This worked in xarray v0.8.2, but no longer works in v0.9.1: ds.to_netcdf(f, engine='scipy') when f is a file-like object.

The traceback looks like: File "...../xarray/core/dataset.py", line 953, in to_netcdf unlimited_dims=unlimited_dims) File "...../xarray/backends/api.py", line 546, in to_netcdf path = _normalize_path(path) File "...../xarray/backends/api.py", line 51, in _normalize_path if is_remote_uri(path): File "...../xarray/core/utils.py", line 439, in is_remote_uri return bool(re.search('^https?\://', path)) File "...../re.py", line 146, in search return _compile(pattern, flags).search(string) TypeError: expected string or buffer

The problem is that to_netcdf does not check to make sure the path is a string before calling _normalize_path: https://github.com/pydata/xarray/blob/b3fc6c4e4fafdf4f075b791594633970a787ad79/xarray/backends/api.py#L545-L548

For now, it's easy enough to work around this by creating a byte-string with to_netcdf and then writing the file separately, but it would be nice to support writing to the file-like object directly again.

With xarray v0.9.1, I get the confusing error message: ValueError: invalid engine for creating bytes with to_netcdf: 'scipy'. Only the default engine or engine='scipy' is supported

This check should be elif engine != 'scipy' instead: https://github.com/pydata/xarray/blob/b3fc6c4e4fafdf4f075b791594633970a787ad79/xarray/backends/api.py#L541

Looking at the doc build from tonight, something happened and netCDF4 isn't getting picked up. All the docs depending on the netCDF4 package are broken (e.g. plotting, IO, etc.).

@shoyer - You may be able to just resubmit the doc build, or maybe we need to fix something.

This makes it possible to read datasets that don't follow CF datetime conventions perfectly, such as the following example which (surprisingly) comes from NCEP/NCAR (you'd think they would follow CF!)

``` ds = xray.open_dataset('http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GEFS/Global_1p0deg_Ensemble/members/GEFS_Global_1p0deg_Ensemble_20150114_1200.grib2/GC') print ds['time'].encoding['units']

u'Hour since 2015-01-14T12:00:00Z' ```

Requirements: - [x] Better documentation: - [x] Tutorial introduces DataArray before and independently of Dataset - [x] Revise README to emphasize that xray generalizes pandas to N-dimensions - [x] New FAQ section to clarify relationship of xray to pandas, Iris and CDAT (#112) - [x] Update What's New - [x] More consistent names: - [x] dimensions -> dims and coordinates -> coords (#190) - [x] noncoordinates -> noncoords - [x] Lay groundwork for non-index coordinates (#197): - [x] Require specifying attrs with a keyword argument in Dataset.__init__ (to make room for coords) - [x] Don't allow indexing array.coords[0] - [x] Remove the to argument from Dataset.apply (it will be much less clearly useful when we have non-index coords) - [x] Add warning in the docs (clarify that "linked dataset variables" are going away)

Nice to have: - [x] Support modifying DataArray dimensions/coordinates in place (#180) - [ ] Automatic alignment in mathematical operations (#184) - [ ] Revised interface for CF encoding/decoding (#155, #175)

``` import cPickle as pickle import xray import numpy as np import pandas as pd

foo_values = np.random.RandomState(0).rand(3,4) times = pd.date_range('2001-02-03', periods=3) ds = xray.Dataset({'time': ('time', times), 'foo': (['time', 'space'], foo_values)})

with open('mypickle.pkl', 'w') as f: pickle.dump(ds, f)

with open('mypickle.pkl') as f: myds = pickle.load(f)

myds ```

This code results in: <repr(<xray.dataset.Dataset at 0x7f95a3290d90>) failed: AttributeError: mapping>