Overall Sunbird-AI-Assistant setup on our Infra

[prathamApp]

Hi,
We are trying to implement the Sunbird AI assistant bot framework to replicate our existing bot. Below are some of the questions we have:

1. Hosting and Service Integration:

• Do we need to host all the services on one VM?
• What should be the minimum configuration required to host these services?
• How should we integrate these services with each other within the Sunbird framework?

Sakhi API-Related Questions:

1. Flow Validation for Hosting the Sakhi API Service

   • Our proposed flow is as follows:
     1. Run the Docker file.
     2. Update the environment variables with our API keys and details.
     3. Install Marqo.
     4. Generate embeddings for our documents using Marqo.
     5. Store the embeddings in Marqo.
     6. Host the Sakhi API using Apache or another web server, then utilize the service.
   • Is this flow correct and complete?

2. Embedding Model and Vectorstore Code

   • We are currently using the OpenAI embedding model. Do we need to update this model in the vector store code?
   • Should we include the embedding model details in the environment file and update the prompts in the config.ini file?

3. Bhashini ULCA API Key Usage

   • We have a Bhashini ULCA API key. Can this be used for speech-to-text, translation, and TTS services within the Sunbird framework?

4. Endpoint Configuration for ASR, TTS, and Translation Services

   • Do we need to specify separate endpoints for ASR, TTS, and translation services?
   • We have ASR, translation, and TTS endpoints for AI4Bharat models. Can these be integrated into the Sunbird framework?

We look forward to your guidance on these matters.

[Jayaprakash8887]

VectorStore: 4Core CPU 16GB RAM
API and whatsapp services: 2 Core CPU 4/8GB RAM minimum.

Based on your load, you can increase the CPU nodes. Increasing RAM will not be necessary.

[Jayaprakash8887]

In order to use openai embeddings, you will have to do code customisation in the sakhi-api-service. Please note that current Marqo version in the service do not support vector embedding. Supports Sentence Transformers only.

You can try with the default setup in the service first and verify the accuracy of results that you are looking for before proceeding with customisation. If results are satisfactory, it will save you openai embeddings generation cost as well as customisation effort

[Jayaprakash8887]

Support for Both Bhashini, AI4Bharath is currently available in release-3.0.0. You can configure the required translation service. Please refer to https://github.com/Sunbird-AIAssistant/sakhi-api-service/blob/release-3.0.0/.env.example