Improve LLM completion timing for edge and node functions

[markbackman]

Issue: #67.

This fix comes in two parts:

• Pipecat: Add a means to optionally defer function call completion pipecat#970
• This Flows change

This PR resolves a timing/race condition issue that resulted in variable and unpredictable timing for how LLM completions occur. This change makes both node and edge function calls predictable in that:

• Edge function calls (which transition to a new node) result in an LLM completion after both the function call result and the next nodes messages are added to the context. This ensures that the LLM responds only once and with the necessary task and context.
• Node function calls (which happen within the existing node) result in an LLM completion once the function call result is added to the context (default Pipecat behavior).

⚠️ This change is breaking in that you need to pass the context_aggregator to the FlowManager at initialization. The benefit for this is warranted though as it results in LLM completions creating a natural and controllable conversation.

For reviewing, the code change is only in the manager.py. All of the other files are changes to align with the FlowManager updates.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
pipecat-flows	✅ Ready (Inspect)	Visit Preview	Jan 14, 2025 0:43am

[markbackman]

Tests are failing because a newer version of Pipecat is required... I'll wait until the 0.0.53 release before merging this due to the dependency.

[filipi87]

LGTM!