issue_comments

I'm not sure if this link will expire, but until it's on youtube, you can watch the talk at https://zoom.us/rec/play/DzaWjz_hMBP23Vqv7T5jPcY1zU4fps2ZL-yAi8MyM5-lbYq-ZQS4ejWMzwxRW53vGu2F1DybYiKSb8M.mYwmkdDSK6ECc8Ux?startTime=1635508803000&_x_zm_rtaid=hMxhM6kwS-ae1hLStT7UXA.1635955310424.1ade0b45b8e3297ff743d3acc0aa08e1&_x_zm_rhtaid=397

Update: hvPlot's .interactive support has been greatly improved and expanded in the new hvPlot 0.7.3 release. It is now showcased at holoviz.org, which introduces how to use hvPlot to build plots, then how to use xarray .interactive and pandas .interactive to add widgets (whether to hvPlot plots or to anything else, including .plot output or tables or xarray reprs). There are still plenty of improvements to make, but apart from documenting .interactive in xarray's docs, I would think this issue can now be closed.

hvPlot's .interactive() support for xarray and pandas was released in in hvPlot 0.7.0 (installable with conda install hvplot=0.7) and is now documented on the website.

There are a few things I think we can still improve (listed at https://github.com/holoviz/panel/issues/1824, https://github.com/holoviz/panel/issues/1826, https://github.com/holoviz/hvplot/issues/531, https://github.com/holoviz/hvplot/issues/533), but it's already really fun to use -- just take your xarray or pandas pipeline da.method1(val1=arg1).method2(val2=arg2,val3=arg3).plot(), add .interactive, and then substitute a Panel widget or ipywidget for any of the arguments: da.interactive.method1(val1=widget1).method2(val2=arg2,val3=widget2).plot()

You can use this with the native .plot() plotting, interactive .hvplot() plots, or pretty much anything you can get out of such a pipeline (table, text, etc.). Try it out and let us know how it goes (here, on one of the issues linked above, or in a new issue at https://github.com/holoviz/hvplot/issues)! Thanks for all the suggestions and ideas here...

The same dataset as in that Visualizing_Meshes notebook is also shown in https://examples.pyviz.org/bay_trimesh, which is more self-contained and may be a better starting point.

Also note that VTK and pyvista are well supported by Panel, as illustrated in various examples at https://panel.holoviz.org/gallery. It would be great to update those examples to include xarray data sources to more fully capture the pipeline from data to display.

I'm not sure what the specific pain points are with UGRID or what specifically is being requested at https://github.com/pyviz-topics/EarthSim/issues/326, but we'd be happy to have examples either in EarthSim or at examples.pyviz.org that start with UGRID datasets and go via Xarray to either 2D or VTK-based 3D visualizations.

That is so cool! I think the syntax is already as good as I can imagine.

Thanks, @philippjfr!

What Philipp outlines above addresses the key limitation that I pointed out previously:

The operations available in HoloViews are only a small subset of what can be done with the native Xarray or Pandas APIs, and adding new capability like that to HoloViews is difficult

As of HoloViews release 1.13.2 that limitation is now completely gone, because a HoloViews interactive operation pipeline can now invoke arbitrary Xarray or Pandas API calls. So you're no longer limited to what has been encapsulated in HoloViews, and you can use the native Xarray method syntax that you're used to. Thus it's now possible to achieve most (all?) of the functionality discussed above, i.e. easily constructing arbitrarily deep Xarray-method pipelines with interactive widgets controlling any step along the way, replaying only that portion of the pipeline when that widget is changed.

So, what's left? As Philipp suggests, we can make the syntax for working with this functionality simpler in hvPlot. At that point we should probably show the syntax required for each of the interactive pipelines demonstrated or suggested in this issue, and see if there's any change to Xarray that would help make the syntax easier or more natural for Xarray users. Either way, the power is now there already!

I didn't appreciate exactly how much of this panel/holoviews can already do

On the one hand, yes, HoloViews + Panel is quite powerful and clean for what it can already do. But just so everyone is on the same page, the workflow @philippjfr shows above is only possible for the operations that HoloViews has implemented internally. The operations available in HoloViews are only a small subset of what can be done with the native Xarray or Pandas APIs, and adding new capability like that to HoloViews is difficult because HoloViews supports many different underlying data formats (lists, dictionaries, NumPy, Pandas, Xarray, etc.). So while there are advantages to what's already available in HoloViews:

• Same syntax for working with a wide variety of data libraries or native Python types
• Easy interactive, reactive pipelines (lazy operations that replay on demand)
• Native support for multiple plotting libraries

there are also major disadvantages:

• You have to learn HoloViews syntax for operations you quite likely already know how to do in your data library of choice (Xarray, Pandas, etc.)
• The supported operations aren't ever going to be as rich as what's available from individual specific libraries

Note that hvPlot injects the plotting capability from HoloViews into Xarray and Pandas, letting you use the native data APIs for plotting, but it doesn't give you the control over lazy/interactive/reactive pipelines that HoloViews' native API offers. So to me what this issue's proposal would entail is taking the idea of hvPlot further, making Xarray (and Pandas) natively act like HoloViews already does -- with lazy operations where interactive controls can be inserted at every stage, letting people stay in their preferred rich, native data API while having the power to easily make anything interactive and to easily make anything visualizable.

I think the difference between what I'm proposing here and what already exists (e.g. in holoviews, xrviz, etc.) is considering interactivity as something that is useful independent of plotting.

The interactive widgets in holoviews and xrviz are obtained from Panel, which is a separate library that is already explicitly designed for specifying and constructing interactivity independent of plotting. E.g. we often use Panel widgets with no plotting to set up simulations or analyses interactively, then run whatever we specified. The interactive function in Panel already works much like what you laid out above, unless I'm missing something.

It sounds like you're hoping for something that is independent of plotting (like Panel) and provides interactive widgets (like Panel) but also has specific support for multidimensional arrays (like HoloViews)? I don't think that's much code, but it could be useful to provide for Xarray in a convenient API.

Looks good to me, and very important (for those of us who care about half a pixel! :-)...