Add support for custom widget layouts

[philippjfr]

This PR is a prototype of a major new feature in parambokeh. It adds a number of Viewable and Layout classes which wrap around bokeh models, parambokeh widgets and holoviews objects and allow laying them out in nested rows and columns and allow them to display themselves (in the same way HoloViews objects do). This allows for far more flexibility for laying out widgets and bokeh plots than the existing mechanisms, i.e. passing in plots to parambokeh.Widgets or defining View parameters and the standard HoloViews layouts.

The approach rearchitects the API quite a bit since the parambokeh.Widgets function now returns these new Viewable objects, however, overall this makes things much simpler and more flexible.

Here is a simple example of laying out two widgets:

widgets2 = parambokeh.Widgets(Test2)

row = parambokeh.layout.Row(widgets, widgets2)
print(type(row))
row

And here is an example of an complex dashboard composed of two sets of parambokeh widgets and a HoloViews plot:

from holoviews.streams import Stream

class Style(Stream):
    
    color = param.ObjectSelector(default='red', objects=['red', 'green', 'blue'])
    
    line_width = param.Number(default=1, bounds=(0, 5))
    

class WidgetStream(Stream):
    
    phase = param.Number(default=0, bounds=(0, np.pi*2))
    
    amplitude = param.Number(default=0.5, bounds=(-1, 1))
    
    frequency = param.Number(default=1, bounds=(1, 10))
    
    style = param.Parameter(precedence=-1)
    
    def plot(self, **kwargs):
        xs = np.linspace(0, np.pi*2, 200)
        options = self.style.contents
        return hv.Curve((xs, np.sin(xs*self.frequency+self.phase)*self.amplitude)).options(**options)

    def view(self):
        return hv.DynamicMap(self.plot, streams=[self, self.style]).options(width=600)
        
style = Style()
stream = WidgetStream(style=style)
widgets = parambokeh.Widgets(stream, callback=stream.event)
style_widgets = parambokeh.Widgets(style, callback=style.event)

row = parambokeh.layout.Row(widgets, style_widgets)
parambokeh.layout.Column(row, stream.view())

This is very much a proof of concept and I'm very open to improving the API here. A full walkthrough through the functionality contained within this PR can be seen here: https://anaconda.org/philippjfr/parambokeh_layouts/notebook

[jbednar]

How about:

(widgets + style_widgets) &  stream.view())

or (with a rule for promotion of streams if that's feasible):

(widgets + style_widgets) & stream)

(Where + is row layout and & is column layout, a convention that could conceivably also be used in HoloViews?)

[philippjfr]

Personally my vote is against such magic in this case, I think people are likely to get confused what is and isn't a HoloViews object and it won't be very transparent what kind of object they end up with.

[jbednar]

That's a reasonable concern, but in this case I would think that it doesn't matter a whole lot, as long as the result will be visualizable, which it should be (adding a HoloViews object to any of this should result in a ParamBokeh row, etc.). Once it's viewable and the representation can be printed, I would think people can figure things out.

[kcpevey]

I haven't played with this yet, but just thinking through how I would apply this in a more complicated grid pattern makes my head hurt. Even the quick example here required me to stop and think about how this was being done. I'm wondering about handling layout in a similar way to say, matplotlib subtplots, e.g. You declare a grid then you declare objects which span a certain number of grid cells. For me and my user-base, that's a common, easily interpreted method.

[philippjfr]

In terms of the low level implementation there is a fundamental reason why a gridspec type pattern will not work well here, a gridspec type pattern is predicated on the idea of defining the grid and then assigning what to plot where. Bokeh on the other hand is predicated on a compositional approach where you define each subplot, give them a certain size and then build up your layout by composing the plots together into rows and columns, i.e. a grid is a row of columns or a column of rows. So at least at the lowest level it has to be implemented in this way.

That all being said we could probably build a gridspec style API on top of this column/row based approach, which resizes the plots you feed it based on the number of grid cells they should span.

[philippjfr]

So I've been thinking about the gridspec approach and playing around with it. It's not entirely straightforward because we need an algorithm that iteratively breaks up the grid into rows and columns depending on the specified layout. In terms of API I'm imagining something like this:

gs = parambokeh.layout.GridSpec(2, 2)
gs[0, 0] = hv.Area([1, 2, 3])     # A
gs[1, 0] = hv.Scatter([1, 2, 3])  # B
gs[:, 1] = hv.Curve([1, 2, 3])    # C

---------------
|   A  |      |
|______|   C  |
|   B  |      |
|______|______|

Unlike matplotlib's GridSpec this works by assignment rather than by indexing for the reasons I discussed above.

[philippjfr]

My vote is that we do implement the gridspec approach although not necessarily in this PR.

[jbednar]

The grid-based layout approach is fine and a good thing to offer. E.g. it seems to work well for Dashing. Still, it's unwieldy for simpler cases where we're essentially building figures rather than dashboards.

I think people are likely to get confused what is and isn't a HoloViews object and it won't be very transparent what kind of object they end up with.

I've been thinking about this further, and I think we should be working towards a goal of having a layout be something that works transparently between HoloViews and ParamBokeh, with either library able to work with the other library's objects smoothly and easily, without ParamBokeh necessarily depending on HoloViews or vice versa. I don't know if that means putting something into Bokeh itself to make this work (as Bokeh Server is the key shared element), or just having conditional imports and some duplicate code. But from a user perspective, I want all these things to be true:

1. It should be easy to compose objects into rows and columns.
2. Such objects should include ParamBokeh widget blocks, HoloViews objects, and probably other things.
3. ParamBokeh shouldn't depend on HoloViews; it makes sense without any appeal to HV.
4. HoloViews probably shouldn't depend on ParamBokeh, though I guess that's less crucial since ParamBokeh presumably just depends on Bokeh, and here we'd be talking about HV's Bokeh support, which depends on Bokeh necessarily.
5. Regardless of the library whose objects I'm working with, I should be able to use the same API for composing objects, which for HoloViews is already + for row-based Layouts (to which I propose adding & for column-based layouts).

As a user, I really don't want to dwell a lot on what comes from which library -- I just want to assemble things into a figure, a dashboard, an app, or whatever I'm working on. If we can have each library provide a set of objects that mix and match in this way, then we're golden!

[philippjfr]

It should be easy to compose objects into rows and columns.

This is true now.

Such objects should include ParamBokeh widget blocks, HoloViews objects, and probably other things.

Also true now, where "other things" includes bokeh models.

HoloViews probably shouldn't depend on ParamBokeh, though I guess that's less crucial since ParamBokeh presumably just depends on Bokeh, and here we'd be talking about HV's Bokeh support, which depends on Bokeh necessarily.

Also true now.

Regardless of the library whose objects I'm working with, I should be able to use the same API for composing objects, which for HoloViews is already + for row-based Layouts (to which I propose adding & for column-based layouts).

I disagree very strongly here. Mixing up HoloViews Layouts, which, while row based, may declare rows and columns, with these new layouts is imo a recipe for horrible confusion. At the very minimum the syntax cannot be the same. Let's take an example here:

hv.Element + hv.Element + hv.Element + hv.Element + hv.Element + parambokeh.Widgets + hv.Element

1. According to you this would return a parambokeh.Row
2. It would actually result in a layout like this:

----------------------------------------------------------------
|   Plot   |   Plot   |   Plot   |   Plot   |        |          |
--------------------------------------------- Widget |   Plot   |
|   Plot   |                                |        |          |
----------------------------------------------------------------

while

hv.Element + hv.Element + hv.Element + parambokeh.Widgets + hv.Element + hv.Element + hv.Element

produces

--------------------------------------------------------------------------------
|   Plot   |   Plot   |   Plot   |   Widget   |   Plot   |   Plot   |   Plot   |
--------------------------------------------------------------------------------

That seems incredibly non-obvious and there will be many other weird cases. Additionally a HoloViews Layout and a parambokeh Row/Column have very different APIs, so while you want this particular API to work transparently across both, a user will immediately get confused if they do:

hv.Element + parambokeh.Widget

because none of the standard HoloViews methods will work on the returned parambokeh.Row. Another weirdness is that as soon as they lay out a parambokeh/bokeh object with their HoloViews object the resulting Row/Column will only render with bokeh, which is again confusing.

My strong preference is therefore not to add such magic, at least in the short term, and make the user be explicit about the objects they are declaring. We absolutely do need to consider how we can combine hv.Layout and more flexible Row/Column based layouts fit into a longer term plan, but introducing this syntactic sugar now will imo not only confuse users but make our job harder down the road, when we decide to tackle this problem at the HoloViews level.

[philippjfr]

@jlstevens and @ceball Would be good if could chime in here if you have opinions about this.

[jlstevens]

I think I agree with Philipp: at the very least I don't want to introduce any more compositional operators such as the proposed &. I like DSLs a lot and that is reflected by the design of both lancet and holoviews but once you start using all these libraries together, things can get quite confusing: even if you make the DSLs largely compatible, in reality you still need to keep a mental model of what object is of what type it is.

Although such syntax is concise for the advanced user who has a very clear mental model of what is going on, this can get very confusing for everyone else. We need to use our compositional operators sparingly at the level where we can get the most power out of them, which imho is at the holoviews and not the parambokeh level.

I say that our goal is to make parambokeh work well with holoviews which does not imply that we have to try to make the API such that it gets confusing between the two projects (which should be kept separate as they are now, unless we get around to doing the major widget refactor in holoviews). Philipp's suggestion of

row = parambokeh.layout.Row(widgets, style_widgets)
parambokeh.layout.Column(row, stream.view())

seems entirely reasonable to me for the typical contexts where this flexibility is really required (primarily dashboards). The gridspec proposal also seems perfectly reasonable to me and I think it would be quite flexible if it can be made to work well.

Lastly, I'll note that I agree that we have to improve how we lay things out (which this PR helps address) but I think we need to have an overall plan for how to improve layouts at the holoviews level before considering adding any new special syntax.

[philippjfr]

Thanks, I obviously agree with all of that.

The main thing that we have to figure out before this PR can be merged is backwards compatibility. In the past parambokeh.Widgets would directly display the widgets, while this change means that it now returns an object, which displays itself and can be composed into a more complex layout. In the server case things are even more different since instead of returning the Document the returned parambokeh.layout.Layout object has a server_doc method to access the Document.

Do we have a major release and break backward compatibility, offer some compatibility mode, or maintain backward compatibility for now and offer a switch to enable the new behavior?

[jbednar]

According to you this would return a parambokeh.Row

Just to be clear, I am not proposing that this would return anything in particular, at least not in my most recent message. I'm just trying to outline a set of principles for how I think things should work. I love what this PR achieves, but the principle here (#5) is that I don't think we should have two totally separate ways of combining things into layouts between ParamBokeh and HoloViews (and maybe Bokeh?). I think we should have one way, used by both libraries, because anything else will perpetually be confusing about what's supported, what limitations there are, and so on. But that comes back to needing to do our own layout to make that work, because of Matplotlib, right? Plus it sounds like we have to wait on the widget refactor to get anywhere close to that?

Sigh. I was hoping we could get some of the way there with this PR, but it sounds like instead it's going to introduce a different, incompatible API, a different way of doing layouts, and a different type of object that can be laid out, when what I think we should be aiming for is a unified way of laying things out (as both HoloViews and Parambokeh equally need the ability to have things in rows and columns, in gridspecs, and so on), unified types of objects that can be laid out between both libraries (not totally separate types that we have to keep track of, as here), and so on. I just want "chunk" and "nested rows and columns of chunks"!

[philippjfr]

I was hoping we could get some of the way there with this PR, but it sounds like instead it's going to introduce a different, incompatible API, a different way of doing layouts, and a different type of object that can be laid out, when what I think we should be aiming for is a unified way of laying things out (as both HoloViews and Parambokeh equally need the ability to have things in rows and columns, in gridspecs, and so on), unified types of objects that can be laid out between both libraries (not totally separate types that we have to keep track of, as here), and so on.

Any replacement for HoloViews' Layout is at least 2-3 months of dedicated work that would likely have to be done by me. I don't foresee myself having that kind of availability anytime soon unless someone were to pay us specifically for that work (which seems reasonably unlikely). So I'd strongly argue that we do not wait on that, although we should decide what we plan on doing there in the long term.

I can work on improving the capabilities in this PR to allow laying out matplotlib and plotly objects alongside the HoloViews, Bokeh and parambokeh objects that are already supported. Secondly, if you think it's a good idea we can move these components into an entirely new package, which parambokeh would then depend on.

Also let me quickly summarize how this pyviz_layouts package would differ from native functionality in bokeh and holoviews:

• Support for laying out bokeh/holoviews/parambokeh/matplotlib/plotly objects
• More flexible layouts than HoloViews but only renders to bokeh output
• Provides thin wrappers to allow all these components to display themselves in the notebook (like HoloViews objects and perhaps mpl/plotly already do)
• Provides comms machinery to allow parambokeh widgets to communicate both in the notebook and on the server
• More generally allows layouts to be easily be displayed and deployed on bokeh server and inside the notebook

I'll also quickly lay out the options for providing an overall vision for layouts (as I see them):

1. Eventually replace holoviews Layout with the bokeh based layouts introduced here, deprecating hv.Layout.

• Pros:
  • Simplifies things massively
  • Good support for dashboard workflows
• Cons:
  • No native layout engine, everything would be based around bokeh layouts
  • Lose ability for cross-layout functionality, e.g. normalization, linking axes, linked brushing etc.
  • Cannot export bokeh layouts using matplotlib's or plotly's save functionality
  • Huge backward incompatibility implications for existing users

2. Do nothing since hv.Layout provides "native" layout engine implemented for each backend, while this PR provides higher level layout engine suited for dashboards

• Pros:
  • Retains ability for cross-layout functionality mentioned in 1.
  • Can export HoloViews based layouts with backend specific save functionality
  • Restrictive HoloViews layouts can be supplemented with more flexible bokeh layouts, which allows building complex dashboards easily
  • Good support for both dashboard and native export workflows
  • No extra work, i.e. ready to use immediately
• Cons:
  • Somewhat duplicative and perhaps confusing

3. Redesign HoloViews Layouts around row/column concept, by subclassing objects introduced in this PR while providing native implementations for matplotlib/plotly in HoloViews:

• Pros:
  • All the pros of 2. except for "No extra work"
  • Greater capabilities for native matplotlib and plotly layouts
  • In transition phase equivalent to 2., until native matplotlib/bokeh implementations are available, then could provide smooth transition from old-style layouts to the new layouts. In other words we would retain hv.Layout, while introducing hv.Column/hv.Row and once we are done with the work the layout operators (+ and potential new operator &/|) could be changed to return Row/Column instead of Layout.
• Cons:
  • Considerable extra work to implement native layout implementations for matplotlib/plotly

Personally I would argue that 1. is completely out of the question, abandoning native matplotlib/plotly layouts would be a huge step back in HoloViews' capabilities. Therefore I argue 2. is the correct thing to do right now, and if we do ever find the time/funding we could extend 2. with 3. by writing native plotting implementations for Row/Column based layouts and thereby replace hv.Layout.

In the short term the functionality in this PR would then become a high-level layout engine meant for building dashboards or complex plots restricted to bokeh based layouts. In the longterm it could then be extended to provide a high-level layout engine which works across backends.

[jlstevens]

I think there is too much to discuss to lay it all out in a github issue: we should have a meeting to decide what to do.

[jbednar]

Yes, we'll meet about it Monday. I can follow most of your description and analysis, but I still have one big question in the meantime:

1. Cons:
   Lose ability for cross-layout functionality, e.g. normalization, linking axes, linked brushing etc.

I don't see how this con follows from option 1. Can't we unify layouts while having some features that apply only to a subset of the "chunks" involved? That's the part I'm not getting; how does allowing everything to mix and match have to hurt any existing functionality? E.g. for normalization and linking axes, can't what we are laying out have a certain level of base functionality that applies to all chunks (large-scale arrangements on the screen), but then some extended functionality that is supported only between some types of chunks?

I.e., if we lay out these items all in a single notebook cell or dashboard:

1. A HoloViews/Bokeh Image
2. A HoloViews/Bokeh Curve
3. A HoloViews/Matplotlib Curve
4. A set of ParamBokeh widgets
5. A Pandas dataframe (in a Div)
6. Some static HTML text (in a Div)

Can't items 1, 2, 3 all be normalized together (before any zooming) regardless of items 4, 5, 6 existing?
Can't items 1, 2 have linked axes and linked brushing regardless of item 3 not being able to be linked?

If we are implementing a new layout system as we are here, I don't see why we necessarily have to give up the features we rely on from HoloViews+Bokeh layouts. Can't those simply be optional features that can be used when needed?

[jbednar]

Ah, I think I misunderstood your option 1. You aren't saying that using the new layout system would have to work in this way, just that it's the way it works right now, and that these would be the consequences of abandoning hv Layouts now. Ok, yes, I agree, we can't do that! And your option 3 sounds essentially like what I'm suggesting in my previous comment, i.e. a generalization of what this PR does, adding special HoloViews-based support to make it achieve feature parity with HoloViews layouts, with an eye to eventually supplanting them. Sounds good.

[jbednar]

pyviz_layouts package

I'm not sure how best to break up packages, but it does seem to me like this functionality is turning into something that makes sense separately from Param and from Bokeh, given the range of types that can be composed. Using the name pyviz does make sense, given what's covered, but maybe focus on apps or dashboards rather than layout, so that people know why they would care about it? Maybe pyviz_dashboards? Or else maybe some unique name that is something we could market separately, e.g. pyviz's tackboard, or pyviz panel.

[jbednar]

I think this elif should have a comment above it saying "Matplotlib figure", as that's not at all obvious from what's being tested.

[jbednar]

Presumably this error message is out of date. Maybe "Not one of the currently supported displayable objects (Bokeh models, HoloViews objects, or Matplotlib figures)."?

Would it be possible to have a fallback that allows any object with a _repr_html_ method? That would cover Pandas .head() and possibly streamz objects, and would let users make their own wrappers around other things...

[philippjfr]

Would it be possible to have a fallback that allows any object with a repr_html method?

Sure.

and possibly streamz objects

Definitely not, those work using ipywidgets.

[jlstevens]

pyviz_dashboards that was one of my first suggestions (well pyviz_dashboard). Not sure about tackboard or panel though I agree that dashboard doesn't entirely convey the concept...