Suggestion: Improved support for Workflow Templates

[sambhav]

Currently it can be fairly verbose to use WorkflowTemplates, can we somehow make it easier to use them such that we get type completion for inputs/outputs a workflow template expects.

We could also potentially investigate auto generating template code from templates already in the cluster.

[sambhav]

The template auto generation can potentially be exposed as a CLI that looks like the following -

hera generate template <template-name> [--namespace <namespace] [--cluster-template]

[flaviuvadan]

fairly verbose to use WorkflowTemplates

Thank you for bringing this up! Do you mind describing a bit more of what makes usage challenging?

type completion for inputs/outputs a workflow template expects

Sounds very interesting! Does this mean something like auto-completion on the fields that a user needs to supply to templates?

The template auto generation can potentially be exposed as a CLI that looks like the following -

Again, very interesting 🚀 would the --namespace be autogenerated by the CLI and showcased to the user as the field that is missing?

[elliotgunton]

As mentioned in #346, we also want to have required Parameters with no default or given values for use in WorkflowTemplates. On

Do you mind describing a bit more of what makes usage challenging?

I can give my suggestions:

Currently using a template looks like:

Task(
    name="email-ref-task",
    template_ref=TemplateRef(
        name="email-workflow-template", template="email", cluster_scope=True
    ),
    inputs=[
        Parameter(
            name="recipients",
            value='["[email protected]"]',
        ),
        Parameter(name="subject", value="Hera test"),
        Parameter(name="body", value="Body of email."),
    ],
)

A typed interface might look like something composed of the following (a bit pseudocode-y to get the point across):

class TemplatedTask(Task):
    """Does some restructuring of how it uses `inputs` to have a nice interface,
    requires a `template_ref`"""

class EmailTemplate(TemplateRef):
    """Clearly shows possible parameters with types, maybe use Pydantic + validate params"""

    recipients: List[str]
    subject: str
    body: str
    some_optional_param: Optional[str]

# Construct your template instance with actual values
email_template_instance = EmailTemplate(
    name="email-workflow-template",
    recipients=my_workflow_recipients,
    subject="Workflow Ran Successfully",
    body="See task run at https://...",
    # Allows us to skip some_optional_param, but skipping any others will fail linter checks, along with
    # any wrong types passed in
)
# Pass template instance to a TemplatedTask which will internally set its `inputs` to the given template_ref's
email_task = TemplatedTask(
    name="email-ref-task", template_ref=email_template_instance
)

[Trollgeir]

I'm not opposed to the idea, but I feel like this is just sugar which requires integrations and a CLI we currently haven't planned for.

I think the first natural step would be to be able to create I/O interfaces for templates in general, and then the next step could perhaps include auto-generations from the cluster. But landing on a good system for I/O interfaces might take some time!

In most cases I'm guessing the user would have the Hera-version of the defined WorkflowTemplate in their repository. Perhaps they could just import its interface in pure python and use it in another Hera workflow?