issue_comments

@jsignell feel free to pick this up, that would be great if you could make this finally happen! (Again, I'm sorry for letting this sit so long).

I'm going to edit the checklist in my 1st comment. There is indeed a lot of things that we can move to follow up issues.

was there any experiment with the html "detail" and "summary" pairs ?

I agree it would highly simplify the HTML code, but when I tried it things were not that easy (I don't remember exactly what, I think it had to do with alignment of nested lists) and I had some weird issues with conflicts between HTML reprs in different output cells. See: https://github.com/jupyterlab/jupyterlab/issues/3200#issuecomment-344012759 and the comment below. Probably I'm missing something obvious?

the css is not injected at load time when the notebook is not trusted

How do Iris and Dask deal with this limitation?

I've quickly checked the related PRs https://github.com/dask/dask/pull/4794 and https://github.com/SciTools/iris/pull/2918. Dask adds style attributes to HTML elements while Iris seems to encapsulate a <style> element in every repr.

Ooh that's nice! Iris and zarr html representations look nice too (i hadn't followed those developments), definitely some good ideas for the xarray html repr! I think the dask and zarr html outputs would integrate very well with the repr here and it would be quite straightforward to encapsulate it in the drop-down html containers of each coordinate / data variable here.

I also like the idea of the summary image like shown above, although this could be harder to achieve.

It seems to me @benbovy that 90% of your ToDo list is nice-to-have or special-case stuff which can be left for later?

Yes, actually most of the work is done. I was mainly worried by how the html repr would look in the different notebook front-ends, but now that other projects (dask, iris, zarr) have such repr, it looks like there's is no major issue. I also struggled with grid column resizing for correctly displaying the variable names, but I think that @shoyer's suggestion https://jsfiddle.net/1g04ykum/ is good enough for now.

is the checklist still up-to-date? The length of it is a bit scary TBH ;-)

Yes it is still up-to-date :-)

But this list is exhaustive and a lot of things could be saved for later! Some of the items are easy to implement but require a decision.

I did a little more tweaking of text-overflow for truncation. This version shows the full name when you hover over it: https://jsfiddle.net/1g04ykum/

Nice!

Hi @benbovy - how can we convince you to work more on this amazing idea? What help / support do you need from other xarray devs?

I'd really like to see this finally happen soon, especially that've I already spend a good amount of time on it (a while ago, I admit). But honestly (and sadly), it's been hard for me to find free time to continue the work on this feature. I'm sorry for that.

I'm also a bit worried by the things (mostly related to compatibility with notebook front-ends and themes) that we'll need to support/fix quickly when this will be ready. Maybe we should make it opt-in for one or two releases.

I would be extremely pleased if anyone is willing to jump in and help on the front-end part (HTML/CSS)! See the checklist at the top of this PR. Unfortunately, my limited expertise in this area makes me rather unproductive.

You can see it by running the xframe example notebook with binder. It actually looks very much like pandas dataframe (with "multi-index" rows for ndims > 2), with some hover effects showing the coordinates names/values at data elements.

The output of xtensor objects is slightly different but interesting too, with nested tables (xtensor's binder). I haven't checked if numpy-html provides the same output.

We could also borrow ideas from https://github.com/agoose77/numpy-html or https://github.com/QuantStack/xframe/commit/90638ecde4d2fd1ae452c8dcd933652e540727a5 for displaying the data of each variable here.

I'll try if we can have good results using fixed columns widths (thus not using display: content)...

I re-implemented the Dataset repr using CSS grid (https://jsfiddle.net/Lmqq7yzz/9/), which I think is much cleaner for column widths that fit the content.

However, one big limitation is that it's currently compatible only in Firefox! Because we want the columns in different sections aligned, I had to define a single grid at the top level and then use display: contents so that all the nested children elements can be positioned using this same grid. Hopefully this will be soon supported in Chrome and Safari (https://caniuse.com/#feat=css-display-contents).

Two other, smaller issues: - Column-width may change on section expand/collapse as apparently it is re-calculated with the visible elements only. This is a bit annoying. - I couldn't get working highlighted rows on hover in this implementation.

Note : in the link above, I changed a bit the design. Variable attributes and data repr can now be show/hidden using clickable icons on the right (tooltips are still needed). This is better from a UX point of view, IMO.

EDIT: tooltips would be also very useful to show full variable names and/or lists of dimensions when these are truncated.

My bigger concern is that it feels hacky and might be slow.

Agreed! Moreover, I have a "Python" icon appearing in the MacOS Dock, which I think it's caused by initializing tk. That's bad!

I played a bit and it seems feasible to estimate an approximate relationship between the text width and the number of characters (see https://gist.github.com/benbovy/fce796c663728b1bdbb3f1514daa458c -- it's a very naive approach, though).

I still haven't gotten a chance to use CSS grid, been hoping for a good moment.

I don't know much about it, but it seems very powerful. That would be the cleanest solution. I'll take a look.

I would think you could achieve this using a table with colspan? I'm not sure about the expandable/hide-able part though.

Yes we could, but I was indeed thinking more about the expandable/hide-able part. With pure html/css the hidden/shown container must be child or sibling of its controller, and I don't know how to achieve that with our current layout design using a table.

I think we should probably avoid adding a tkinter dependency. I would rather assume a fixed column-width for the first column.

Even considering that tkinter is already shipped with CPython as part of the standard library? My concern with an arbitraily fixed column-width is that it should be wide enough to cover a reasonable range of use cases, but when the variable names are really short (it occurs often in examples, e.g., 'foo', 'x', 'y'...) it won't look very nice (I haven't tested it yet, though). I guess we can also calculate the width by hand considering the worst case scenario in order to have a good margin...

Something like this with a table? [...] You'll have to embed some JS to do it though

That solution (JS included) would be nice if we can support all notebook front-ends without any extra installation or configuration step.

The sizing of variable name and dimensions columns according to their content is tricky because (1) we want these columns be aligned between different sections (Coordinates and Data variables) and also aligned with the list of dimension labels in the Dimension section, (2) there are subsections for variable attributes and data repr, which both take 100 % width and (3) we want the sections and subsections to be collapsible/expandable but we are limited by a pure html/css solution.

While using <table> is usually appropriate for rendering that kind of content, for the reasons above I don't think it is possible here, unfortunately. Unless someone has a better idea, I don't see any other option than calculating a fixed size for the columns before rendering.

But I don't see neither any robust way to calculate these sizes. One option could be to use tkinter, e.g.,

```python

import tkinter as tk from tkinter import font tk.Tk() root = tk.Tk() front_end_font = font.Font(family='Helvetica', size=11, weight='bold') front_end_font.measure("variable_name") 76 root.destroy() ```

That's not very elegant to say the least, but it has the advantage of being part of the Python standard library. The problem is that we don't know the font-family and font-size. We could define it explicitly in the CSS code but it's better to inherit it from the notebook front-ends (in some cases it is dynamically defined, e.g., the jupyterlab presentation mode). So a workaround might be to use a common font which has wide characters to calculate the width + add a good safety margin.

If anyone has a better idea, e.g., a layout using some kind of smart CSS grid system... that would be great!

@rgbkrk? @ellisonbg?