Great, I'll look at that implementation. Thanks!

We'd probably have to write a so called HoloViews DataInterface for scipp. See the equivalent xarray implementation: https://github.com/holoviz/holoviews/blob/master/holoviews/core/data/xarray.py

Just been sent a link to this discussion after having worked on something very similar for our project (which resembles Xarray in many ways): https://github.com/scipp/scipp/pull/2573 I am now wondering if we could somehow use the .interactive approach for our needs instead.

@philippjfr how much work would it be to implement an .interactive method for our own classes? Our DataArray is slightly different from Xarray's. Thanks!

Sophia Yang and I wrote a blog post about hvplot interactive. It's based on Pandas dataframes but it works the same way for Xarray. Check it out https://towardsdatascience.com/the-easiest-way-to-create-an-interactive-dashboard-in-python-77440f2511d1

You can also find the repo and links to binder+colab here https://github.com/sophiamyang/hvplot_interactive

I meant to at this link to the PyData Talk on .interactive including video https://discourse.holoviz.org/t/pydata-2021-build-polished-data-driven-applications-directly-from-your-pandas-or-xarray-pipelines/3017/4

Just for completeness. You can find @philippjfr PyData 2021 .interactive talk here https://pydata.org/global2021/schedule/presentation/51/build-polished-data-driven-applications-directly-from-your-pandas-or-xarray-pipelines/. Quite powerful.

Inspired by that I've created a gist here https://gist.github.com/MarcSkovMadsen/e666503df2aa1d8d047dcb9555b5da6d. It's for a pandas DataFrame. But the principle is the same for xarray.

https://user-images.githubusercontent.com/42288570/140560629-818083cc-838a-41f4-8908-7a66791d8ce6.mp4

@TomNicholas I've been playing around with an interactive accessor, very much an experiment for now (and requires some small fixes in HoloViews) but I think this could be heading in the right direction:

https://anaconda.org/philippjfr/xarray_interactive/notebook

Having taken the ideas presented here as inspiration the latest HoloViews release actually extends what we had described above and provides the capability to use arbitrary xarray methods to transform the data and control the parameters of those transforms using Panel based widgets. The HoloViews docs show one such example built on xarray which is built around so call dim expressions:

```python import panel as pn import xarray as xr

air_temp = xr.tutorial.load_dataset('air_temperature')

We declare a dim expression which uses the quantile method from the xr namespace

and provides a panel FloatSlider as the argument to the expression

q = pn.widgets.FloatSlider(name='quantile') quantile_expr = hv.dim('air').xr.quantile(q, dim='time')

We now wrap the xarray Dataset in a HoloViews one, apply the dim expression and cast the result to an image

temp_ds = hv.Dataset(air_temp, ['lon', 'lat']) transformed = temp_ds.apply.transform(air=quantile_expr).apply(hv.Image)

Now we display the resulting transformation pipeline alongside the widget

pn.Column(q, transformed.opts(colorbar=True, width=400)) ```

I am likely to integrate this capability with hvPlot with a more intuitive API, e.g. in this case I'd expect to be able to spell this something like this:

python xrds = xr.tutorial.load_dataset('air_temperature') q = pn.widgets.FloatSlider(name='quantile') quantile_expr = hv.dim('air').xr.quantile(q, dim='time') xrds.hvplot.image(transforms={'air': quantile_expr})

I didn't appreciate exactly how much of this panels/holoviews can already do - I think I need to go away and experiment with using/wrapping them but aiming for an xarray-like syntax.

Maybe wait until early next week when I anticipate new Panel and HoloViews releases to be out which smooth out some issues with these workflows.

One thing I didn't mention above is that in the pipeline I showed HoloViews will cache the intermediate changes so that if you change the color or change the resampling frequency it only executes the part of the pipeline downstream from where the parameter changed.

I think the real power in this proposal is in the ability to chain operations on interactive components using an API that will be familiar to xarray users. We have a similar concept in HoloViews which allows you to build complex processing and visualization pipelines. I'll work through some examples in HoloViz ecosystem to show what is possible there and maybe provide some ideas or approaches that might work here.

Let's work with a relatively contrived but simple example and load the air_temperature sample dataset:

python airtemps = xr.tutorial.open_dataset('air_temperature') ds = hv.Dataset(airtemps)

In this example you explode your dataset into individual chunks for each longitude, then apply a reduction along the latitude and finally cast the output to a Curve giving us a Curve of the mean temperature at each longitude:

python curves = ds.groupby('lon', dynamic=True).apply.reduce(lat=np.mean).apply(hv.Curve).opts(width=600, framewise=True)

Now we decide we want to resample the data too, so we import the resample operation and apply it to our existing pipeline:

from holoviews.operation.timeseries import resample resample(curves, rule='7d')

But really we don't just want to compute the mean we want to pick the reduce function and we also want to be able to set the resampling frequency and pick a color. By combining Panel and HoloViews you can inject widget parameters at every stage:

```python function = pn.widgets.Select(name='Function', options={'mean': np.mean, 'min': np.min, 'max': np.max}) color = pn.widgets.ColorPicker(name='Color', value='#000000') rule = pn.widgets.TextInput(name='Rule', value='7d')

obj = (ds.groupby('lon', dynamic=True) .apply.reduce(lat=function) .apply(hv.Curve) .apply.opts(width=600, color=color, framewise=True) .apply(resample, rule=rule) )

hv_pane = pn.pane.HoloViews(obj)

pn.Row( hv_pane[0], pn.Column(*hv_pane[1][0], function, color, rule) ) ```

So this shows pretty clearly how useful this kind of chaining/pipeline building can be, especially when built on top of an API like xarray which allows for very powerful data manipulation. I don't have enough of a perspective to say how feasible it would be to implement something like this that comprehensively wraps xarray's API but I'd certainly love to see it. Whether it is built on Panel (which I am of course partial to as the author) or ipywidgets or even supporting both.

My main comments therefore are about the API, it is not clear to me based on what you have said so far which parts of the API are actually interactive, e.g. in this case:

python ida = da.interactive.isel(lat=50, lon=60) ida = (ida - ida.mean('time')).std(dim='time')

Is only sel/isel ever interactive or can other methods also be interactively set? If the answer is no then that's all clear enough and the scope relatively narrow but well defined. If however you intend the entire API (or at least some well defined subset of it) to be interactive then I think there should be some explicit way to declare which parts are interactive and where the values are coming from (and what the values should be if they can't be automatically determined). In the HoloViews example I showed above you explicitly supply widgets but if you don't want users to deal with manually laying things out then you could also just let the user supply the specification of the valid values. Something like in your first example:

python interactive.isel(da, plot_mean_over_time, time=slice(100, 500))

but expanded to include support for discrete lists of items, explicit widgets, and so on.

Hope that's at all helpful! I think the idea is really neat and it could be very powerful indeed.

This looks really cool and I like the API! I'll have to give it a try to give more detailed feedback. Note that I'm not a core developer of xarray but I also think this is best managed as an external project.

Just wanted to ask some clarification on some of your comments.

Some downstream plotting libaries (such as @hvplot) already use widgets when interactively plotting xarray-derived data structures, but they don't seem to go the full N dimensions.

What do you mean by this? hvPlot does let you explore n-dimensional data using widgets, what is the limitation you were seeing there?

This also isn't something that should be confined to plotting functions - you often choose slices or variables at the start of analysis, not just at the end.

This is a good point, but I guess I'm not yet entirely clear on how your proposed APIs would deal with this.