[Feature]: Grounding on Google AI Studio

[traderpedroso]

The Feature

Dear LiteLLM Team,
I am writing to request the addition of support for grounding on Google Web Search when interacting with the Gemini 2.0 Flash model through LiteLLM. This functionality is now available via the v1alpha and v1beta on Google AI Studio endpoint of the Generative Language API: https://generativelanguage.googleapis.com/v1alpha/models/gemini-2.0-flash-exp.

{

  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "best stock for 2025"
        }
      ]
    }
  ],
  "systemInstruction": {
    "role": "user",
    "parts": [
      {
        "text": "Searching on google, answer"
      }
    ]
  },
  "tools": [
    {
      "googleSearch": {}
    }
  ],
  "safetySettings": [
    {
      "category": "HARM_CATEGORY_HARASSMENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_HATE_SPEECH",
      "threshold": "BLOCK_NONE"
    },
    {
      "category": "HARM_CATEGORY_CIVIC_INTEGRITY",
      "threshold": "BLOCK_NONE"
    }
  ]
}

Motivation, pitch

Currently, LiteLLM provides a fantastic abstraction layer for accessing various LLMs, including Gemini models. However, the ability to ground responses with real-time information from the web is crucial for many applications, particularly those requiring:
Up-to-date Information: LLMs have a knowledge cutoff, and grounding with web search allows them to access the most recent data, ensuring more accurate and relevant responses.
Fact Verification: Grounding enables LLMs to verify the information they generate, reducing the risk of hallucinations and providing users with more trustworthy outputs.
Enhanced Contextual Understanding: By leveraging external information, LLMs can better understand the nuances of a user's query and provide more comprehensive and context-aware answers.
Real-World Applications: Many use cases, such as research, content creation, and customer support, require access to external information, making grounding an essential feature.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response