Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add AWS Nova multimodal llm for support realtime #559

Closed
wants to merge 0 commits into from

Conversation

zhuermu
Copy link

@zhuermu zhuermu commented Jan 8, 2025

Add a addon for support Voice Agent Realtime such as: Gemini2.0. User input voice frame and video frame and output voice and text.
The aws Nova multimodal support file,image and video as input output text. So we base on the extension of the Ten framework: agora_rtc+STT+Nova+TTS and we can achieve real-time like effect.
Here are the test results:
image

@plutoless
Copy link
Contributor

i think the bedrock_v2v_python is still consuming input text and produce output text. what is different from the old bedrock_llm_python extension? or should we just update the existing extension to use your code?

Copy link
Contributor

@plutoless plutoless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the pr! but i'm bit confused as the graph tells bedrock_v2v_python is still a text-in text-out llm extension. if needed i would suggest to make your bedrock_v2v_python a updated version of bedrock_llm_python. does this make sense?

@zhuermu
Copy link
Author

zhuermu commented Jan 11, 2025

thanks for the pr! but i'm bit confused as the graph tells bedrock_v2v_python is still a text-in text-out llm extension. if needed i would suggest to make your bedrock_v2v_python a updated version of bedrock_llm_python. does this make sense?

The new bedrock Nova is multimodal. I agree your idea, I try to update the bedrock_llm_python. Thanks .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants