Introduces fanout, rework execution model, further additions

[merschformann]

Description

Changes the step execution model to a simple thread pool based one and introduces fanout.

Changes

• Uses a simpler thread-based execution engine for better control and data exchange between steps.
  • We are aware of Python's parallel CPU utilization limitations due to the GIL, but are looking forward to newer Python releases removing the limitation. The advantages appear to outweigh the disadvantages long-term.
• Adds own thread-pool implementation for full control.
• Introduces @foreach and @join for dynamic fanout.
• Adds support for defining app run options from results of steps via AppRunConfig.
• Adds support for getting the full nextmv.cloud.AppResult (including run metadata) instead of just the "output" (full_result=True).
• Prepares graph state communication.
• Introduces a pipeline config.
• Improves readmes.
• Reduces unnecessary time when waiting for runs.

Preview

This change allows modeling pipelines that dynamically fanout. E.g., the following pipeline spins up 3 app runs for by cloning the input but modifying the app parameter per run.

class Flow(FlowSpec):
    @foreach()  # Run the successor step for each item in the result list of this step
    @step
    def prepare(data: dict):
        """
        Creates 3 copies of the input and configures them for 3 different app parameters.
        """
        inputs = [copy.deepcopy(data) for _ in range(3)]
        run_configs = [AppRunConfig(input, [AppOption("param", i)]) for i, input in enumerate(inputs)]
        return run_configs

    @app(app_id="echo")
    @needs(predecessors=[prepare])
    @step
    def solve():
        """
        Runs the model.
        """
        pass

    @needs(predecessors=[solve])
    @join()  # Collect the results from the previous 'foreach' step and combine them into a list passed as the arg
    @step
    def merge(results: list[dict]):
        """Merges the results."""
        return results

Resolves ENG-5614, ENG-5642