Unit Test Results

6 files           6 suites   55m 20s :stopwatch: 16 230 tests 14 495 :heavy_check_mark: 1 735 :zzz: 0 :x: 90 576 runs  82 392 :heavy_check_mark: 8 184 :zzz: 0 :x:

Results for commit fc8252e8.

:recycle: This comment has been updated with latest results.

@lewisjarednz fyi

@shoyer can I bother you again now that CI is passing please?

Ignoring failing CI due to fsspec (see https://github.com/pydata/xarray/pull/5615#issuecomment-881679900)

xarray/tests/test_backends.py::test_open_fsspec appears to fail because of the release of ffspec 2021.7.0 so this PR will probably have to wait until a fix for that is added (presumably elsewhere to keep the changes clear).

Something like 59ed7d5? (Obviously missing proper tests but just to get a sense of whether the idea is plausible)

I tried pushing a fix. It's unclear to me whether the change should be in how the dtypes are inferred (given that the inference code seems to do what it is meant to...) or whether is_unicode_dtype simply needs to be updated to know about np.str_ (which is the fix I just tried).

What problem are you trying to solve here?

Somehow I ended up with np.str_ in a pandas dataframe (how is unclear to me but this seems to be a valid string type), which then exploded when I converted to xarray and attempted to save as netCDF. Minimal example below.

```python import numpy as np import pandas as pd

I don't know how the strings ended up being np.str_....

scenarios = [np.str_(v) for v in ["scenario_a", "scenario_b", "scenario_c"]] years = range(2015, 2100 + 1) tdf = pd.DataFrame( data=np.random.random((len(scenarios), len(years))), columns=years, index=scenarios, ) tdf.index.name = "scenario" tdf.columns.name = "year" tdf = tdf.stack() tdf.name = "tas"

txr = tdf.to_xarray()

raises error shown below

txr.to_netcdf("test.nc")

error

Traceback (most recent call last): File "scratch.py", line 20, in <module> txr.to_netcdf("test.nc") File ".../lib/python3.7/site-packages/xarray/core/dataarray.py", line 2741, in to_netcdf return dataset.to_netcdf(args, *kwargs) File ".../lib/python3.7/site-packages/xarray/core/dataset.py", line 1699, in to_netcdf invalid_netcdf=invalid_netcdf, File ".../lib/python3.7/site-packages/xarray/backends/api.py", line 1108, in to_netcdf dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims File ".../lib/python3.7/site-packages/xarray/backends/api.py", line 1154, in dump_to_store store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims) File ".../lib/python3.7/site-packages/xarray/backends/common.py", line 256, in store variables, check_encoding_set, writer, unlimited_dims=unlimited_dims File ".../lib/python3.7/site-packages/xarray/backends/common.py", line 294, in set_variables name, v, check, unlimited_dims=unlimited_dims File ".../lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 464, in prepare_variable variable, self.format, raise_on_invalid_encoding=check_encoding File ".../lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 131, in get_datatype datatype = _nc4_dtype(var) File ".../lib/python3.7/site-packages/xarray/backends/netCDF4.py", line 154, in _nc4_dtype raise ValueError(f"unsupported dtype for netCDF4 variable: {var.dtype}") ValueError: unsupported dtype for netCDF4 variable: object ```

I agree, this should totally work. It's not obvious to me how to best fix it, though.

I assume it's not as trivial as just changing e.g. https://github.com/pydata/xarray/blob/f9a535c130615fe81379d785dffcc6c814fc1126/xarray/coding/strings.py#L32 to also know about np.str_?

@shoyer any further thoughts on this now that the scope is clearer?

So far I've just added a single test which fails. I don't think the test should fail although I'm not sure what the np.str_ type actually is so maybe this isn't a bug? Help/advice greatly appreciated.