What is your issue?

I noticed that open_dataset with chunks="auto" fails when netCDF4 variables/coordinates are encoded as NC_STRING. The reason is that xarray reads netCDF4 NC_STRING as object type, and dask cannot estimate the size of a object dtype.

As a workaround, the user must currently rewrite the netCDF4 and specify the string DataArray(s) encoding(s) as a fixed-length string type (i.e "S2" if max string length is 2) so that the data are written as NC_CHAR and xarray read it back as byte-encoded fixed-length string type.

Here below I provide a reproducible example

``` import xarray as xr import numpy as np

Define string datarray

arr = np.array(["M6", "M3"], dtype=str) print(arr.dtype) # <U2 da = xr.DataArray(data=arr, dims=("time")) data_vars = {"str_arr": da}

Create dataset

ds_nc_string = xr.Dataset(data_vars=data_vars)

Set chunking to see behaviour at read-time

ds_nc_string["str_arr"] = ds_nc_string["str_arr"].chunk(1) # chunks ((1,1),)

Write dataset with NC_STRING

ds_nc_string["str_arr"].encoding["dtype"] = str ds_nc_string.to_netcdf("/tmp/nc_string.nc")

Write dataset with NC_CHAR

ds_nc_char = xr.Dataset(data_vars=data_vars) ds_nc_char["str_arr"].encoding["dtype"] = "S2" ds_nc_char.to_netcdf("/tmp/nc_char.nc")

When NC_STRING, chunks="auto" does not work when string are saved as

--> NC STRING is read as object, and dask can not estimate chunk size !

If chunks={} it reads the NC_STRING array in a single dask chunk !!!

ds_nc_string = xr.open_dataset("/tmp/nc_string.nc", chunks="auto") # NotImplementedError ds_nc_string = xr.open_dataset("/tmp/nc_string.nc", chunks={}) # Works ds_nc_string.chunks # chunks (2,)

With NC_CHAR, chunks={} and chunks="auto" works and returns the same result!

ds_nc_char = xr.open_dataset("/tmp/nc_char.nc", chunks={})
ds_nc_char.chunks # chunks (2,) ds_nc_char = xr.open_dataset("/tmp/nc_char.nc", chunks="auto") ds_nc_char.chunks # chunks (2,)

NC_STRING is read back as object

ds_nc_string = xr.open_dataset("/tmp/nc_string.nc", chunks=None) ds_nc_string["str_arr"].dtype # object

NC_CHAR is read back as fixed length byte-string representation (S2)

ds_nc_char = xr.open_dataset("/tmp/nc_char.nc", chunks=None) ds_nc_char["str_arr"].dtype # S2 ds_nc_char["str_arr"].data.astype(str) # U2 ```

Questions: - open_dataset should not take care of automatically deserializing the NC_CHAR fixed-length byte-string representation into a Unicode string? - open_dataset should not take care of automatically reading NC_STRING as Unicode string (converting object to str)?

Related issues are: - https://github.com/pydata/xarray/issues/7652 - https://github.com/pydata/xarray/issues/2059 - https://github.com/pydata/xarray/pull/7654 - https://github.com/pydata/xarray/issues/2040

What happened?

If cmap.set_over is specified, the array color mapping and the colorbar behave badly if the array does not contain values above the norm.vmax.

Let's take an array and apply a colormap and norm (see code below) Now, if in the array I change the array values larger than the norm.vmax (the 2 bottom right pixels) with other values inside the norm: - Using matplotlib I get the expected results - Using xarray I get this weird behavior.

What did you expect to happen?

The colorbar should not "shift" and the array should be colormapped correctly This is possibily related also to https://github.com/pydata/xarray/issues/4061

Minimal Complete Verifiable Example

```Python import matplotlib.colors import numpy as np import xarray as xr import matplotlib as mpl import matplotlib.pyplot as plt

Define DataArray

arr = np.array([[0, 10, 15, 20], [ np.nan, 40, 50, 100], [150, 158, 160, 161], ]) lon = np.arange(arr.shape[1]) lat = np.arange(arr.shape[0])[::-1] lons, lats = np.meshgrid(lon, lat) da = xr.DataArray(arr, dims=["y", "x"], coords={"lon": (("y","x"), lons), "lat": (("y","x"), lats), } ) da

Define colormap

color_list = ["#9c7e94", "#640064", "#009696", "#C8FF00", "#FF7D00"] levels = [0.05, 1, 10, 20, 150, 160] cmap = mpl.colors.LinearSegmentedColormap.from_list("cmap", color_list, len(levels) - 1) norm = mpl.colors.BoundaryNorm(levels, cmap.N) cmap.set_over("darkred") # color for above 160 cmap.set_under("none") # color for below 0.05 cmap.set_bad("gray", 0.2) # color for nan

Define colorbar settings

ticks = levels cbar_kwargs = {
'extend': "max",

}

Correct plot

p = da.plot.pcolormesh(x="lon", y="lat", cmap=cmap, norm=norm, cbar_kwargs=cbar_kwargs) plt.show()

Remove values larger than the norm.vmax level

da1 = da.copy() da1.data[da1.data>=norm.vmax] = norm.vmax - 1 # could be replaced with any value inside the norm

With matplotlib.pcolormesh [OK]

p = plt.pcolormesh(da1["lon"].data, da1["lat"], da1.data, cmap=cmap, norm=norm) plt.colorbar(p, **cbar_kwargs) plt.show()

With matplotlib.imshow [OK]

p = plt.imshow(da1.data, cmap=cmap, norm=norm) plt.colorbar(p, **cbar_kwargs) plt.show()

With xarray.pcolormesh [BUG]

--> The colorbar shift !!!

da1.plot.pcolormesh(x="lon", y="lat", cmap=cmap, norm=norm, cbar_kwargs=cbar_kwargs) plt.show()

With xarray.imshow [BUG]

--> The colorbar shift !!!

da1.plot.imshow(cmap=cmap, norm=norm, cbar_kwargs=cbar_kwargs, origin="upper") ```

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [x] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment