issues

What happened?

I wrapped np.fft.fft with xr.apply_ufunc, and wanted to add coordinates. Here I used np.fft.fftfreq and as input to that function a value derived from the original coordinates of the array passed in to xr.apply_ufunc. This derived value, a float, was still an xarray object.

When then using this value in combination with my result from the call to xr.apply_ufunc, the function apply_variable_ufunc got called again somehow and raised

What did you expect to happen?

It should not call apply_variable_ufunc again.

Minimal Complete Verifiable Example

```Python import xarray as xr import numpy as np

def fft( a: xr.DataArray, dim: str, newdim: str, n: int = None, norm: str = None, keep_attrs=None, ) -> xr.DataArray:

with_dask = a.chunks is not None
func = dask.array.fft.fft if with_dask else np.fft.fft

kwargs = {
    "n": n,
}

if with_dask and norm is not None:
    raise ValueError("norm is not supported with dask arrays")
else:
    kwargs["norm"] = norm

result = xr.apply_ufunc(
    func,
    a,
    kwargs=kwargs,
    input_core_dims=(
        (dim,),
    ),
    exclude_dims=set([dim]),
    output_core_dims=(
        (newdim,),
    ),
    vectorize=False,
    # dask="allowed" if with_dask else "forbidden",
    # output_dtypes=[np.complex128],
    # keep_attrs=_keep_attrs(keep_attrs),
)

# DFT size
n = n if n is not None else _get_length(a, dim)

# Coordinate spacing along `dim`
# convert to float to fix the issue!
delta = (a.coords[dim].diff(dim=dim).mean(dim=dim))
# print(delta)

# Coordinates for `newdim`
result[newdim] = np.fft.fftfreq(n, delta)
return result

duration = 10.0 fs = 8000.0 nsamples = int(fs * duration) f = 100.0 A = 2.0 n = 100 dim = "time" newdim = "frequency" signal = A * np.sin(2.0 * np.pi * f * np.arange(nsamples) / fs) signal = xr.DataArray(signal, dims=[dim], coords={dim: np.arange(nsamples)}).expand_dims("channel") result = fft(signal, n=n, dim=dim, newdim=newdim) ```

MVCE confirmation

• [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

```Python ======================================================================================================================================== FAILURES ========================================================================================================================================= ___________________ testfft _____________________

def test_fft():
    duration = 10.0
    fs = 8000.0
    nsamples = int(fs * duration)
    f = 100.0
    A = 2.0
    n = 100
    dim = "time"
    newdim = "frequency"
    signal = A * np.sin(2.0 * np.pi * f * np.arange(nsamples) / fs)
    signal = xr.DataArray(signal, dims=[dim], coords={dim: np.arange(nsamples)}).expand_dims("channel")

  result = xarray_scipy.signal.fft(signal, n=n, dim=dim, newdim=newdim)

tests/test_signal.py:323:

xarray_scipy/signal.py:187: in fft result[newdim] = np.fft.fftfreq(n, delta) /nix/store/9mr36pjhxnlvdy00njp8dj22rd7n3yg3-python3-3.10.5-env/lib/python3.10/site-packages/numpy/fft/helper.py:169: in fftfreq return results * val ../xarray/xarray/core/arithmetic.py:70: in array_ufunc return apply_ufunc( ../xarray/xarray/core/computation.py:1132: in apply_ufunc return apply_dataarray_vfunc( ../xarray/xarray/core/computation.py:271: in apply_dataarray_vfunc result_var = func(*data_vars)

func = <ufunc 'multiply'>, signature = _UFuncSignature([(), ()], [()]), exclude_dims = frozenset(), dask = 'allowed', output_dtypes = None

def apply_variable_ufunc(
    func,
    *args,
    signature,
    exclude_dims=frozenset(),
    dask="forbidden",
    output_dtypes=None,
    vectorize=False,
    keep_attrs=False,
    dask_gufunc_kwargs=None,
):
    """Apply a ndarray level function over Variable and/or ndarray objects."""
    from .variable import Variable, as_compatible_data

    first_obj = _first_of_type(args, Variable)

    dim_sizes = unified_dim_sizes(
        (a for a in args if hasattr(a, "dims")), exclude_dims=exclude_dims
    )
    broadcast_dims = tuple(
        dim for dim in dim_sizes if dim not in signature.all_core_dims
    )
    output_dims = [broadcast_dims + out for out in signature.output_core_dims]

    input_data = [
        broadcast_compat_data(arg, broadcast_dims, core_dims)
        if isinstance(arg, Variable)
        else arg
        for arg, core_dims in zip(args, signature.input_core_dims)
    ]

    if any(is_duck_dask_array(array) for array in input_data):
        if dask == "forbidden":
            raise ValueError(
                "apply_ufunc encountered a dask array on an "
                "argument, but handling for dask arrays has not "
                "been enabled. Either set the ``dask`` argument "
                "or load your data into memory first with "
                "``.load()`` or ``.compute()``"
            )
        elif dask == "parallelized":
            numpy_func = func

            if dask_gufunc_kwargs is None:
                dask_gufunc_kwargs = {}
            else:
                dask_gufunc_kwargs = dask_gufunc_kwargs.copy()

            allow_rechunk = dask_gufunc_kwargs.get("allow_rechunk", None)
            if allow_rechunk is None:
                for n, (data, core_dims) in enumerate(
                    zip(input_data, signature.input_core_dims)
                ):
                    if is_duck_dask_array(data):
                        # core dimensions cannot span multiple chunks
                        for axis, dim in enumerate(core_dims, start=-len(core_dims)):
                            if len(data.chunks[axis]) != 1:
                                raise ValueError(
                                    f"dimension {dim} on {n}th function argument to "
                                    "apply_ufunc with dask='parallelized' consists of "
                                    "multiple chunks, but is also a core dimension. To "
                                    "fix, either rechunk into a single dask array chunk along "
                                    f"this dimension, i.e., ``.chunk({dim}: -1)``, or "
                                    "pass ``allow_rechunk=True`` in ``dask_gufunc_kwargs`` "
                                    "but beware that this may significantly increase memory usage."
                                )
                dask_gufunc_kwargs["allow_rechunk"] = True

            output_sizes = dask_gufunc_kwargs.pop("output_sizes", {})
            if output_sizes:
                output_sizes_renamed = {}
                for key, value in output_sizes.items():
                    if key not in signature.all_output_core_dims:
                        raise ValueError(
                            f"dimension '{key}' in 'output_sizes' must correspond to output_core_dims"
                        )
                    output_sizes_renamed[signature.dims_map[key]] = value
                dask_gufunc_kwargs["output_sizes"] = output_sizes_renamed

            for key in signature.all_output_core_dims:
                if key not in signature.all_input_core_dims and key not in output_sizes:
                    raise ValueError(
                        f"dimension '{key}' in 'output_core_dims' needs corresponding (dim, size) in 'output_sizes'"
                    )

            def func(*arrays):
                import dask.array as da

                res = da.apply_gufunc(
                    numpy_func,
                    signature.to_gufunc_string(exclude_dims),
                    *arrays,
                    vectorize=vectorize,
                    output_dtypes=output_dtypes,
                    **dask_gufunc_kwargs,
                )

                # todo: covers for https://github.com/dask/dask/pull/6207
                #  remove when minimal dask version >= 2.17.0
                from dask import __version__ as dask_version

                if LooseVersion(dask_version) < LooseVersion("2.17.0"):
                    if signature.num_outputs > 1:
                        res = tuple(res)

                return res

        elif dask == "allowed":
            pass
        else:
            raise ValueError(
                "unknown setting for dask array handling in "
                "apply_ufunc: {}".format(dask)
            )
    else:
        if vectorize:
            func = _vectorize(
                func, signature, output_dtypes=output_dtypes, exclude_dims=exclude_dims
            )

    result_data = func(*input_data)

    if signature.num_outputs == 1:
        result_data = (result_data,)
    elif (
        not isinstance(result_data, tuple) or len(result_data) != signature.num_outputs
    ):
        raise ValueError(
            "applied function does not have the number of "
            "outputs specified in the ufunc signature. "
            "Result is not a tuple of {} elements: {!r}".format(
                signature.num_outputs, result_data
            )
        )

    output = []
    for dims, data in zip(output_dims, result_data):
        data = as_compatible_data(data)
        if data.ndim != len(dims):

          raise ValueError(
                "applied function returned data with unexpected "
                f"number of dimensions. Received {data.ndim} dimension(s) but "
                f"expected {len(dims)} dimensions with names: {dims!r}"
            )

E ValueError: applied function returned data with unexpected number of dimensions. Received 1 dimension(s) but expected 0 dimensions with names: ()

../xarray/xarray/core/computation.py:746: ValueError ```

Anything else we need to know?

No response

Environment

What happened?

Evaluating the coefficients from a polyfit that happened to be none results in a polyval evaluation of zeros.

What did you expect to happen?

I expect the result to be nan when an input is nan as well. Note passing in skipna=True to polyfit does not make a difference.

Minimal Complete Verifiable Example

Python import xarray as xr import numpy as np arr = xr.DataArray(np.ones(100)*np.nan, dims=["time"], coords={"time": np.arange(100)}) xr.polyval(arr.time, arr.polyfit(dim="time", deg=1))

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [ ] Complete example — the example is self-contained, including all data and the text of any traceback.
• [ ] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

No response

Environment

The open_dataset and open_mfdataset functions allow a user to pass in a (custom) backend directly. This was however not functioning.

Selecting an engine with a string was functioning as expected, however, passing in a backend class directly failed.

An instance of the backend is now created. A test has been added, as well as some type hints.

• [ ] Closes #xxxx
• [x] Tests added
• [ ] Passes pre-commit run --all-files
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst

Is your feature request related to a problem? Please describe. In the case of dependent dimensions, there is a lot of missing data, and using a stacked layout is preferable. Composing an array using concat and then stack is not very memory efficient and results in NaN's that have to be removed.

I am now composing an array using concat and then reshaping using stack. This can consume a lot of memory, and requires explicit removal of the NaN's after stacking. Having a stack parameter to concat that takes the desired index would be very useful.

Describe the solution you'd like A stack parameter to concat that takes the desired index would be very useful.

Initially it may just do the naive concat followed by a stack and removal of NaN, but eventually it should insert items correctly without creating NaN.

Describe alternatives you've considered Composing an array using concat and then stack is not very memory efficient and results in NaN's that have to be removed.

Additional context Issue related to concat and stack https://github.com/pydata/xarray/issues/981.

test_reduce in xray.test.test_variable.TestVariable fails for i686 with all Python versions.

``` FAIL: test_reduce (xray.test.test_variable.TestVariable)

Traceback (most recent call last): File "/tmp/nix-build-python3.4-xray-0.6.0.drv-0/xray-0.6.0/xray/test/test_variable.py", line 707, in test_reduce self.assertVariableIdentical(v.mean('x'), v.reduce(np.mean, 'x')) File "/tmp/nix-build-python3.4-xray-0.6.0.drv-0/xray-0.6.0/xray/test/init.py", line 122, in assertVariableIdentical assert as_variable(v1).identical(v2), (v1, v2) AssertionError: (<xray.Variable (y: 3)> array([ 0.48885496, 0.57610051, 0.33430722]), <xray.Variable (y: 3)> array([ 0.48885496, 0.57610051, 0.33430722])) ```

The build log for Python 3.3: http://hydra.nixos.org/build/25641209/nixlog/1/raw The build logs for other Python versions can be found at: http://hydra.nixos.org/eval/1221294?filter=xray&compare=1221225&full=1