Visualise a pipeline based on the YAML definition

[drazendee]

It would be useful to be able to generate a graph of the pipeline based on the YAML definition. It would be easier to understand if my edges are setup correctly if I could see the order that Valohai thinks it would execute the node in.

[skogstrom]

Is the issue that we are not visualizing each edge separately?

[drazendee]

You'd want a cli tool to draw a graph. Right now the only way to see it is to adhoc run or go through git and UI (but for dev stage this is a bit tedious)

[ruksi]

so the goal is for the CLI to render the pipeline graph

some ideas:

1. simple but the furthest from the actual: create the graph using one of ASCII terminal rendering libraries, should work for a quick visualization to debug; there are many small libraries but I haven't encountered any that would be above the rest 😛
2. battle-tested but a bit old-school: use Graphviz + Python bindings to render the graph locally to an image to pop open
3. most similar to what it will look like, but a bit more complex: use Cytoscape to render it like our app does but just show it in a local browser or as a rendered image, would probably benefit from extracting the rendering code to a separate library so it can be used in both projects. This approach would also enable interactivity if that is something that is desired, but also requires the most from the computer running the CLI 🤔

[akx]

We could also add a view in the app that allows visualizing an arbitrary pipeline graph passed in as base64'd JSON query parameter (and have the CLI open that); that way the rendering would look familiar.

[ruksi]

We could also add a view in the app that allows visualizing an arbitrary pipeline graph passed in as base64'd JSON query parameter (and have the CLI open that); that way the rendering would look familiar.

🤩 that would something "best of both worlds" as connecting to Valohai App is probably fine (as CLI surely needs to do that anyway)