Pipeline init taking around 10 seconds with daily + rtvi

[Vaibhav-Lodha]

RTVI + Daily.co transport Issue

I am using gemini bot example but with TTS + STT + LLM instead of gemini multi-modal but for some reason the entire init process is taking 10+ seconds in each run and not able to figure out where can i save time, i am thinking to try threading instead of process but i believe that wont change much of things.

Pipecat Version - 0.0.52

Python - 3.12

2025-01-21 17:37:31.576 | INFO     | server:lifespan:56 - Preloaded templates
2025-01-21 17:37:45.894 | INFO     | auth:verify_frontend_cors_origin:62 - Request Received with origin: origin='http://localhost:5173'
2025-01-21 17:37:45.894 | INFO     | auth:verify_frontend_cors_origin:65 - Local environment detected. Skipping CORS origin verification: origin='http://localhost:5173'
2025-01-21 17:37:45.894 | INFO     | auth:verify_frontend_api_key:27 - Request Received
2025-01-21 17:37:45.894 | INFO     | auth:verify_frontend_api_key:30 - Local environment detected. Skipping API Key verification.
2025-01-21 17:37:45.903 | DEBUG    | demo.daily_webrtc.api.bots:connect:44 - Received connect request with params: conversation_id='70faca0c-057f-4222-b442-7770a35633f2' actions=[] bot_profile=<AllowedBotProfiles.ARJUN: 'Arjun'> customer_details=CustomerDetails(name='[REDACTED]', email='[REDACTED]', phone='[REDACTED]', created_at=datetime.datetime(2025, 1, 21, 17, 37, 45, 903359)) bot_type=<BotType.CHAT: 'chat'>
2025-01-21 17:37:45.903 | DEBUG    | demo.daily_webrtc.api.bots:connect:58 - Creating room for conversation 70faca0c-057f-4222-b442-7770a35633f2 at 0.00s
2025-01-21 17:37:45.903 | INFO     | utils.dailyco_utils:bot_create_room_and_user_token:15 - Creating room for bot [REDACTED]
2025-01-21 17:37:45.904 | INFO     | utils.dailyco_utils:bot_create_room_and_user_token:23 - Creating room for bot at 0.00030937499832361937
2025-01-21 17:37:47.055 | INFO     | utils.dailyco_utils:bot_create_room_and_user_token:36 - Created room [REDACTED] for bot at 1.1521163339930354
2025-01-21 17:37:47.465 | INFO     | utils.dailyco_utils:bot_create_room_and_user_token:40 - Got user token for room [REDACTED] at 1.5611999590037158
2025-01-21 17:37:47.465 | DEBUG    | demo.daily_webrtc.api.bots:connect:62 - Created room [REDACTED] with user token [REDACTED] at 1.56s
2025-01-21 17:37:47.465 | INFO     | lead_details_api:save_to_redis:173 - Saving lead details to Redis: name='[REDACTED]' email='[REDACTED]' phone='[REDACTED]' created_at=datetime.datetime(2025, 1, 21, 17, 37, 45, 903359)
2025-01-21 17:37:47.467 | INFO     | lead_details_api:save_to_redis:179 - Lead details saved successfully to Redis: name='[REDACTED]' email='[REDACTED]' phone='[REDACTED]' created_at=datetime.datetime(2025, 1, 21, 17, 37, 45, 903359)
2025-01-21 17:37:47.467 | DEBUG    | demo.daily_webrtc.api.bots:connect:78 - Returning room URL [REDACTED] and user token [REDACTED] at 1.56s
2025-01-21 17:37:47.468 | INFO     | demo.daily_webrtc.bot:bot_launch:108 - Launching bot for room [REDACTED]
2025-01-21 17:37:47.468 | INFO     | utils.dailyco_utils:create_room_token_for_bot:49 - Creating bot token for room [REDACTED] at [REDACTED]
2025-01-21 17:37:47.468 | INFO     | utils.dailyco_utils:create_room_token_for_bot:57 - Getting bot token for room [REDACTED] at 0.00010912500147242099
2025-01-21 17:37:48.438 | INFO     | utils.dailyco_utils:create_room_token_for_bot:61 - Got bot token for room [REDACTED] at 0.96924695900816
2025-01-21 17:37:48.438 | INFO     | demo.daily_webrtc.bot:bot_launch:111 - Starting a new bot process for room [REDACTED]
2025-01-21 17:37:50.517 | INFO     | demo.daily_webrtc.bot:_bot_process:98 - Inside of bot process for room [REDACTED]
2025-01-21 17:37:50.517 | INFO     | demo.daily_webrtc.bot:_bot_main:76 - Running _bot_main for room [REDACTED]
2025-01-21 17:37:50.517 | INFO     | demo.daily_webrtc.bot:create_task:52 - Using bot pipeline chat_bot_pipeline for room room_url='[REDACTED]' with params=BotParams(conversation_id='70faca0c-057f-4222-b442-7770a35633f2', actions=[], bot_profile=<AllowedBotProfiles.ARJUN: 'Arjun'>, customer_details=CustomerDetails(name='[REDACTED]', email='[REDACTED]', phone='[REDACTED]', created_at=datetime.datetime(2025, 1, 21, 17, 37, 45, 903359)), bot_type=<BotType.CHAT: 'chat'>)
2025-01-21 17:37:50.517 | INFO     | demo.daily_webrtc.chat_bot_pipeline:chat_bot_pipeline:25 - Creating bot pipeline for room room_url='[REDACTED]' with params=BotParams(conversation_id='70faca0c-057f-4222-b442-7770a35633f2', actions=[], bot_profile=<AllowedBotProfiles.ARJUN: 'Arjun'>, customer_details=CustomerDetails(name='[REDACTED]', email='[REDACTED]', phone='[REDACTED]', created_at=datetime.datetime(2025, 1, 21, 17, 37, 45, 903359)), bot_type=<BotType.CHAT: 'chat'>) with params.bot_profile=<AllowedBotProfiles.ARJUN: 'Arjun'>
2025-01-21 17:37:50.517 | INFO     | demo.daily_webrtc.chat_bot_pipeline:chat_bot_pipeline:30 - Using agent config: agent_name='Arjun' prompt='demo_collections' max_session_time=300 stt=<STTServiceEnum.deepgram: 'deepgram'> stt_config=STTConfig(model=None, sample_rate=16000, params=None) llm=<LLMServiceEnum.openai_default: 'openai_default'> llm_config=LLMConfig(model=None, params=None) tts=<TTSServiceEnum.elevenlabs_flash: 'elevenlabs_flash'> tts_config=TTSConfig(sample_rate=24000, voice=None, model=None, params=None) tools=[<ToolEnum.END_CALL: 'end_call'>]
2025-01-21 17:37:50.520 | INFO     | lead_details_api:get_lead_details_from_exotel_event:216 - from_number='[REDACTED]' with call_data={'from': '[REDACTED]'}
2025-01-21 17:37:50.524 | INFO     | lead_details_api:get_lead_details_from_exotel_event:229 - Lead details: {'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'}, call_data={'from': '[REDACTED]'}, lead_details_with_from_num={'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'} lead_details_with_to_num=None
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking PipelineSource#0 -> DailyInputTransport#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking DailyInputTransport#0 -> RTVIProcessor#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking RTVIProcessor#0 -> OpenAIUserContextAggregator#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking OpenAIUserContextAggregator#0 -> OpenAILLMService#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking OpenAILLMService#0 -> RTVIBotLLMProcessor#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking RTVIBotLLMProcessor#0 -> RTVIBotTranscriptionProcessor#0
2025-01-21 17:37:50.532 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking RTVIBotTranscriptionProcessor#0 -> DailyOutputTransport#0
2025-01-21 17:37:50.533 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking DailyOutputTransport#0 -> OpenAIAssistantContextAggregator#0
2025-01-21 17:37:50.533 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking OpenAIAssistantContextAggregator#0 -> PipelineSink#0
2025-01-21 17:37:50.533 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking Source#0 -> Pipeline#0
2025-01-21 17:37:50.533 | DEBUG    | pipecat.processors.frame_processor:link:150 - Linking Pipeline#0 -> Sink#0
2025-01-21 17:37:50.533 | DEBUG    | pipecat.pipeline.runner:run:27 - Runner PipelineRunner#0 started running PipelineTask#0
2025-01-21 17:37:50.533 | INFO     | pipecat.transports.services.daily:join:322 - Joining [REDACTED]
2025-01-21 17:37:51.298 | INFO     | pipecat.transports.services.daily:on_participant_joined:620 - Participant joined [REDACTED]
2025-01-21 17:37:51.299 | INFO     | lead_details_api:get_lead_details_from_exotel_event:216 - from_number='[REDACTED]' with call_data={'from': '[REDACTED]'}
2025-01-21 17:37:51.300 | INFO     | lead_details_api:get_lead_details_from_exotel_event:229 - Lead details: {'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'}, call_data={'from': '[REDACTED]'}, lead_details_with_from_num={'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'} lead_details_with_to_num=None
2025-01-21 17:37:51.300 | INFO     | demo.daily_webrtc.chat_bot_pipeline:on_participant_joined:131 - Chat Bot: greetings={'name': 'assistant', 'role': 'assistant', 'content': 'Hello, Good Evening. This is Arjun from ICSC Bank. Am I speaking with [REDACTED]?'}
2025-01-21 17:37:51.300 | INFO     | demo.daily_webrtc.bot_pipeline_runner:_on_participant_joined:61 - Participant joined.
2025-01-21 17:37:53.202 | INFO     | pipecat.transports.services.daily:join:340 - Joined [REDACTED]
2025-01-21 17:37:53.293 | DEBUG    | demo.daily_webrtc.bot_pipeline_runner:_on_app_message:75 - Received app message: message={'data': {}, 'label': 'rtvi-ai', 'type': 'client-ready', 'id': 'e30258a4'} - sender='[REDACTED]'
2025-01-21 17:37:53.293 | INFO     | demo.daily_webrtc.chat_bot_pipeline:on_client_ready:83 - Clients are ready, setting bot ready.
2025-01-21 17:37:53.294 | INFO     | demo.daily_webrtc.chat_bot_pipeline:on_client_ready:86 - Bot is ready, params.actions=[]
2025-01-21 17:37:54.294 | INFO     | lead_details_api:get_lead_details_from_exotel_event:216 - from_number='[REDACTED]' with call_data={'from': '[REDACTED]'}
2025-01-21 17:37:54.294 | INFO     | lead_details_api:get_lead_details_from_exotel_event:229 - Lead details: {'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'}, call_data={'from': '[REDACTED]'}, lead_details_with_from_num={'name': '[REDACTED]', 'email': '[REDACTED]', 'phone': '[REDACTED]', 'created_at': '2025-01-21 17:37:45'} lead_details_with_to_num=None
2025-01-21 17:37:54.295 | INFO     | demo.daily_webrtc.chat_bot_pipeline:on_client_ready:94 - Voice Bot: greetings={'name': 'assistant', 'role': 'assistant', 'content': 'Hello, Good Evening. This is Arjun from ICSC Bank. Am I speaking with [REDACTED]?'}```

[Vaibhav-Lodha]

Hey @markbackman ,

I noticed that the daily.co APIs seem to be a bit slow for the Indian region. We’re hosting our servers in Mumbai, and each API call, like creating a room, getting a token, or joining a room, takes around 1 second or more. Do you have any idea why this might be happening?

[markbackman]

@Vaibhav-Lodha, the REST APIs are hosted in us-west-2, so that would explain some of the latency (~170ms roundtrip). Note that all media is geolocated, so once your room is created, your connecting to a media server in Mumbai. That still doesn't explain the 10 second delay.

Is this repeatable?

[Vaibhav-Lodha]

Yes @markbackman it happens in every event, and have also pointed the room creation to mumbai still no help :(

[markbackman]

There is no room creation in Mumbai. Room creation is always us-west-2. We have team members in India and Asia Pacific. I'll see if they encounter a similar issue.

[Vaibhav-Lodha]

@markbackman did we get a chance to have a word with indian team?

[markbackman]

He did run the demo, but used Daily Prebuilt. I've asked him to run again using an RTVI client, so that the data is more like-for-like.