User configurable visualisations

[markusdregi]

A discussion on user configurable visualisations was discussed some time back. It is still into the future as our priorities are towards other milestones at this time, but I'll create this issue as a placeholder for the discussion.

[markusdregi]

From equinor/ert#775:

During the workshop introduction we discussed different "phases":

Short-term: ERT-webviz visualization through ERT workflow (or something equivalent) at the end of a run.
(Semi-)long-term: ERT run is started/configured through web front-end, and data is visualized as they come in/becomes available through the ERT API (being either after e.g. each iteration, or even more live for some data types, like certain forward models, or run status).
We want to allow the user to be able to configure also the visualization setup. At the same time we don't want configuration of the visualization setup to be required (e.g. ERT provides a default setup). One question is then how to merge (optional) user visualization configuration input, with ERT default visualization setup. Roughly there are perhaps two main options:

User provides (an optional) webviz/visualization configuration at start of ERT run (e.g. at the same time as the ERT config itself is provided). ERT merges this with ERT default webviz configuration (see below)?
ERT outputs a default webviz configuration after the run. User changes this based on his/her wishes/needs.
Short-term both options could work, however the first one might be the preferred one. Benefits include:

User can configure ERT (incl. visualization) once (no need of merging ERT default output with user requests after each run).
Long-term, user can then include custom user visualizations, but still using more "live" data from e.g. custom ERT forward models to populate them (e.g. after each iteration), compared to only showing data after finished run.
If going for alternative 1, one remaining main question is then how to achieve

ERT should provide a default setup (in case of no user visualization setup input)
If user wants to provide input/custom content, he/she still probably wants to use ERT standard visualization components/dashboards.
Which components/dashboards that the user wants to include (both ERT standard plugins, and custom user plugins) might depend on ERT run purpose (ensemble experiment, AHM, sensitivity run...)
Two possibilities here includes:

ERT automatically adds one or more pages to the user input webviz configuration, with standard ERT plugins.
User has responsibility of explicitly adding requested ERT plugins to the configuration, if he/she wants to override default configuration.
Regarding conditional inclusion/exclusion of components, based on ERT run purpose, one possibility to consider is perhaps to allow for jinja2 statements in the visualization configuration (but this should follow the same user experience as will be chosen for the new ERT yml configuration file - but there are indications that jinja2 will be used when replacing the current ERT configuration setup?).
Using Dash wildcards (which is a Dash feature just around the corner), we could probably get the inclusion/exclusion probably done with Dash/webviz features alone.

One aim is perhaps to let the equinor/ert repository be a ert cli/ert api repository (kind of like it is today), and then e.g. another repository equinor/ertviz is some optional building block (containing dash, webviz-config... as extra dependencies) which calls the ert cli/ert api. Then users will then always have the option of using ert without dash/webviz.

↑ Everything above is just some suggestions/initial thoughts - very much up for changes/input.