issue_comments

Just for completeness. You can find @philippjfr PyData 2021 .interactive talk here

Oh awesome! Can I watch this talk anywhere? That link just seems to have a summary.

@jbednar that all looks amazing! Can't wait to properly try it out.

Given that much of what I imagined is now available in holoviews, I will close this issue now. But if you would like to raise a PR pointing towards this functionality somewhere in xarray's docs (maybe either as a more detailed description in the Ecosystem page or as a note in the plotting page of the user guide) then that would be welcome!

@philippjfr that looks incredible!

The accessor syntax is exactly what I was imagining too, great job.

requires some small fixes in HoloViews

I would love to have a go, plus I had a few other ideas I would like to try out - is there a branch somewhere I could check out to get it going locally?

This looks absolutely great @philippjfr ! I would be keen to help you and @jbednar with making the syntax as intuitive and familiar as possible for xarray users. If you have any relevant issues/PR's in holoviews or here then please tag me :)

It sounds like you're hoping for something that is independent of plotting (like Panel) and provides interactive widgets (like Panel) but also has specific support for multidimensional arrays (like HoloViews)? I don't think that's much code, but it could be useful to provide for Xarray in a convenient API.

Thanks @jbednar , I think that's a good summary of most of what I was imagining.

I think the real power in this proposal is in the ability to chain operations on interactive components using an API that will be familiar to xarray users.

Yes exactly. There will be a lot of users who do their work in xarray and being able to achieve interactivity in their existing workflows with almost exactly the same API would improve their experience without presenting much of a barrier to adoption.

Thanks for the (impressive) example @philippjfr !

I think there should be some explicit way to declare which parts are interactive

I was imagining that functions/methods following the .interactive accessor was the only place where interactivity occurred, but it might well be possible to do it more generally than that and still keep it intuitive.

I didn't appreciate exactly how much of this panels/holoviews can already do - I think I need to go away and experiment with using/wrapping them but aiming for an xarray-like syntax.

Thanks @dcherian , I hadn't seen those.

I think the difference between what I'm proposing here and what already exists (e.g. in holoviews, xrviz, etc.) is considering interactivity as something that is useful independent of plotting.

The aim would be to allow interactive parameterization of arbitrary functions, which could (and often would) be plotting functions, but could actually be anything. That way analysis can be interactively parameterized, and the plotting can be handled by any library. (Plotting libraries could also choose to reuse these interactivity functions, but wouldn't have to.) I think that approach would integrate well with being able to change plotting backends too (#3553).

This looks really cool and I like the API!

Great!

I'll have to give it a try to give more detailed feedback.

Thanks, but it's definitely not ready for that yet, I'll post here and tag you when it is.

What do you mean by this? hvPlot does let you explore n-dimensional data using widgets, what is the limitation you were seeing there?

I had a go with hvPlot's gridded data classes and although it worked well for plotting variation along one dimension with a single slider, I got some errors when I tried to plot N-D data with multiple slider widgets along more than one dimension. It looks like that might have been user error though... I'll compare more closely and raise issues if necessary.

I'm not yet entirely clear on how your proposed APIs would deal with this.

I'm referring to the discussion on method chaining: that proposed API (using an InteractiveDataArray) would allow you to interactively select a subset of data python ida = da.interactive.isel(lat=50, lon=60) before specifying the analysis to perform on it python ida = (ida - ida.mean('time')).std(dim='time') and an ida.plot() or compute call on the same object later would still be tied to the original sliders. That's quite different to only being able to create the sliders in the final call.

IMHO, I would rather see this maintained in a separate project

Yeah that's a fair point. I think this is another case where the ecosystem of packages orbiting xarray could do with being more explicitly organised.

Reasons for direct integration in xarray: - Availability to all users: Functionality should be of general interest to anyone using xarray with jupyter, it's not domain-specific at all, - Makes writing robust code a bit easier because can then rely on private xarray methods for parsing indexers and so on

Reasons for a separate xarray-interactive repository: - Keeps developer maintenance / issue tracking separate - If plotting library-specific interfaces are desired they can be adding without cluttering main repo

I guess either way I could just write it in a separate repo and if in future we decided to include it in xarray master then move it.

@philippjfr @rabernat would be interested in your perspectives as developers/users of these downstream libraries? Would this be useful or not really?

(also I realise that the suggestion at the end is similar to a task graph of dask.delayed objects, but I assume something will go wrong if I try to wrap dask arrays with xarray dataarrays with dask delayed?)

Difficulties with method chaining

Arbitraily long method chaining would be great, i.e. python da.interactive.isel(time=10).mean('time').plot() but I think it will be considerably more complicated.

The problem is that the way the ipywidgets.interactive() function works means that each time a widget value is altered (e.g. a slider dragged to a new position), then the function wrapped by interactive must be recomputed. For single functions that's fine, but for method chaining it means the final .plot() method has to know about all the previous methods back up to the .interactive input.

I've found a way to get around this, but I'd like some feedback on the approach because it might be needlessly complicated.

I would like to do it by subclassing to create an InteractiveDataArray, which you could create with an interactive accessor method like python ida = da.interactive.isel(time=10) This class would store the widgets and decorate it's inherited methods to either propagate them (e.g. through ida.reduce()) or display them (e.g. after ida.plot()). It would define the _ipython_display_() method so that calling display(ida) revealed the widgets.

To allow for the final method to recompute all the previous steps, each inherited computation method would be wrapped by a decorator which records the function used and it's arguments. That way the final method (which really you know will either be .plot(), or __print__()) can revaluate it's whole history when the slider tells it to recompute.

I've got a very rough example of this working, but as I said there might be a much easier way...