According to the open_dataset documentation, using chunks="auto" or chunks={} should yield datasets with variables chunked depending on the preferred chunks of the backend. However neither the netcdf4 nor the h5netcdf backend seem to implement the preferred_chunks encoding attribute needed for this to work.

This PR adds this attribute to the encoding upon data reading. This results in chunks="auto" in open_dataset returning variables with chunk sizes multiples of the chunks in the nc file, and for chunks={}, returning the variables with then exact nc chunk sizes.

• [x] Closes #1440
• [x] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

Code Sample

```python import xarray as xr import numpy as np import dask.array as da

arr = np.arange(25).reshape((5, 5))

l_indices = xr.DataArray(np.array(((0, 1), (2, 3))), dims=['lines', 'cols']) c_indices = xr.DataArray(np.array(((1, 3), (0, 2))), dims=['lines', 'cols'])

xarr = xr.DataArray(da.from_array(arr, chunks=10), dims=['y', 'x'])

print(xarr[l_indices, c_indices])

xarr[l_indices, c_indices] = 2 ```

Problem description

This crashes on the last line with a IndexError: Unlabeled multi-dimensional array cannot be used for indexing: [[0 1] [2 3]] I'm expecting to be able to do assignment this way, and it doesn't work.

Expected Output

Expected output is the modified array with 2's in the indicated positions

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr import numpy as np

arr = xr.DataArray(np.ones((5, 5)), attrs={'units': 'K'}) xr.set_options(keep_attrs=True) arr

<xarray.DataArray (dim_0: 5, dim_1: 5)>

array([[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.]])

Dimensions without coordinates: dim_0, dim_1

Attributes:

units: K

arr.clip(0, 1)

<xarray.DataArray (dim_0: 5, dim_1: 5)>

array([[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.],

[1., 1., 1., 1., 1.]])

Dimensions without coordinates: dim_0, dim_1

```

Expected Output

I would expect the attributes to be kept

Problem Description

keep_attrs set to True doesn't seem to be respected with the DataArray.clip method.

Output of xr.show_versions()

MCVE Code Sample

```python import xarray as xr import numpy as np arr2 = xr.DataArray(np.ones((2, 2)), dims=['y', 'x']) arr1 = xr.DataArray(np.ones((2, 2)), dims=['y', 'x'], coords={'y': [0, 1], 'x': [0, 1]})

del arr1.coords['y'] del arr1.coords['x']

shows arr1 without coordinates

arr1

shows coordinates in xarray 0.15

arr1 * arr2

```

Expected Output

python <xarray.DataArray (y: 2, x: 2)> array([[1., 1.], [1., 1.]]) Dimensions without coordinates: y, x

Problem Description

In xarray 0.15, the coordinates are restored when doing the multiplication: python <xarray.DataArray (y: 2, x: 2)> array([[1., 1.], [1., 1.]]) Coordinates: * y (y) int64 0 1 * x (x) int64 0 1

Output of xr.show_versions()

MCVE Code Sample

python import xarray as xr from weakref import ref arr = xr.DataArray([1, 2, 3]) ref(arr)

Expected Output

I expect the weak reference to be created as in former versions

Problem Description

The above code raises the following exception: TypeError: cannot create weak reference to 'DataArray' object

Output of xr.show_versions()

• [x] Closes #2563
• [x] Tests added (for all bug fixes or enhancements)
• [x] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Code Sample, a copy-pastable example if possible

A "Minimal, Complete and Verifiable Example" will make it much easier for maintainers to help you: http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports

```python import xarray as xr

from netCDF4 import Dataset

def write_netcdf(filename,zlib,least_significant_digit,data,dtype='f4',shuffle=False,contiguous=False,\ chunksizes=None,complevel=6,fletcher32=False): file = Dataset(filename,'w') file.createDimension('n', 1) foo = file.createVariable('data',\ dtype,('n'),zlib=zlib,least_significant_digit=least_significant_digit,\ shuffle=shuffle,contiguous=contiguous,complevel=complevel,fletcher32=fletcher32,chunksizes=chunksizes) foo[:] = data file.close()

write_netcdf("mydatafile.nc",True,None,0.0,shuffle=True, chunksizes=(1,))

data = xr.open_dataset('mydatafile.nc') arr = data['data']

arr[0].to_netcdf('mytestfile.nc', mode='w', engine='h5netcdf')

```

Problem description

The above example crashes with a TypeError since xarray 0.10.4 (works before, hence reporting the error here and not in eg. h5netcdf): TypeError: Scalar datasets don't support chunk/filter options

The problem here is that it is not anymore possible to squeeze an array that comes from a netcdf file that was compressed or filtered.

Expected Output

The expected output is that the creation of the trimmed netcdf file works.

Output of xr.show_versions()

Hi,

While trying to support DataArrays in pyresample, I stumble upon what seems to me to be a bug. It looks like unstacking a dimension takes unreasonable amounts of memory. For example:

```python from xarray import DataArray import numpy as np

arr = DataArray(np.empty([1, 8996, 9223])).stack(flat_dim=['dim_1', 'dim_2']) print(arr) arr.unstack('flat_dim') ```

peaks at about 8GB of my memory (in top), while the array in itself isn't supposed to take more than 635MB approximately. I know my measuring method is not very accurate, but should it be this way ?

As a side note, the unstacking also takes a very long time. What is going on under the hood ?

Martin

Code Sample, a copy-pastable example if possible

```python import numpy as np import xarray as xr

arr = xr.DataArray([[1, 2, 3]], dims=['time', 'x']) arr['time'] = np.array([1]) time_bnds = xr.DataArray([0, 1], dims='time_bounds') arr['time'].attrs['bounds'] = time_bnds

dataset = xr.Dataset({'arr': arr, 'time_bnds': time_bnds})

dataset.to_netcdf('time_bnd.nc') ```

Problem description

This code produces a TypeError Traceback (most recent call last): File "test_time_bounds.py", line 12, in <module> dataset.to_netcdf('time_bnd.nc') File "/home/a001673/.local/lib/python2.7/site-packages/xarray/core/dataset.py", line 1132, in to_netcdf unlimited_dims=unlimited_dims) File "/home/a001673/.local/lib/python2.7/site-packages/xarray/backends/api.py", line 598, in to_netcdf _validate_attrs(dataset) File "/home/a001673/.local/lib/python2.7/site-packages/xarray/backends/api.py", line 121, in _validate_attrs check_attr(k, v) File "/home/a001673/.local/lib/python2.7/site-packages/xarray/backends/api.py", line 112, in check_attr 'files'.format(value)) TypeError: Invalid value for attr: <xarray.DataArray (time_bounds: 2)> array([0, 1]) Dimensions without coordinates: time_bounds must be a number string, ndarray or a list/tuple of numbers/strings for serialization to netCDF files This is a problem for me because we need to provide attributes to the coordinate variables and save the to netcdf in order to be CF compliant. There are workarounds (like saving the time_bnds as a regular variable and putting its name as an attribute of the time variable) , but the provided code seems to be the most intuitive way to do it.

Expected output

I would expect an output like this (ncdump -h): netcdf time_bnd { dimensions: time = 1 ; time_bounds = 2 ; x = 3 ; variables: int64 time(time) ; time:bounds = "time_bnds" ; int64 time_bnds(time_bounds) ; int64 arr(time, x) ;

Output of xr.show_versions()

Code Sample, a copy-pastable example if possible

```python import numpy as np import h5netcdf

filename = "mask_and_scale_float32.nc"

with h5netcdf.File(filename, 'w') as f: f.dimensions = {'x': 5} v = f.create_variable('hello', ('x',), dtype=np.uint16) v[:] = np.ones(5, dtype=np.uint16) v[0] = np.uint16(65535) v.attrs['_FillValue'] = np.uint16(65535) v.attrs['scale_factor'] = np.float32(2) v.attrs['add_offset'] = np.float32(0.5)

import xarray as xr

v = xr.open_dataset(filename, mask_and_scale=True)['hello'] print(v.dtype) ```

Problem description

The scale_factor and add_offset being float32, I would expect the result from loading to be a float32 array. However, we get a float64 array instead. A float32 array for a very large dataset is better for faster computations.

Expected Output

float32

Output of xr.show_versions()