Google Gemini 2.0 #62
Conversation
| @@ -23,7 +23,7 @@ lazy_static! { | |||
| let region = std::env::var("GOOGLE_REGION").unwrap_or("us-central1".to_string()); | |||
| let project_id = std::env::var("GOOGLE_PROJECT_ID").expect("PROJECT_ID not set"); | |||
|
|
|||
| format!("https://{}-aiplatform.googleapis.com/v1/projects/{}/locations/{}/publishers/google/models/gemini-pro:streamGenerateContent?alt=sse", | |||
| format!("https://{}-aiplatform.googleapis.com/v1/projects/{}/locations/{}/publishers/google/models", | |||
There was a problem hiding this comment.
The URL is not model-specific so I had to implement that in model methods.
| @@ -284,11 +284,11 @@ pub struct GoogleGeminiProDate { | |||
| #[derive(Debug, Serialize, Deserialize)] | |||
| pub struct GoogleGeminiProUsageMetadata { | |||
| #[serde(rename = "promptTokenCount")] | |||
| pub prompt_token_count: i32, | |||
| pub prompt_token_count: Option<i32>, | |||
There was a problem hiding this comment.
For some reason Vertex does not always return token counts, so had to make it optional.
| Gemini2_0ProExpVertex, | ||
| Gemini2_0FlashThinkingExpVertex, | ||
| // Legacy models | ||
| #[deprecated( |
There was a problem hiding this comment.
All the deprecated models.
| Some(GoogleModels::Gemini2_0FlashThinkingExpVertex) | ||
| } | ||
| // Gemini 1.0 Pro is deprecated starting 2/15/2025. We are re-routing to 1.5 Pro for the model | ||
| "gemini-pro" => Some(GoogleModels::Gemini1_5Pro), |
There was a problem hiding this comment.
As per comment, automatically re-routing to 1.5 for deprecated models.
| @@ -132,119 +330,72 @@ impl LLMModel for GoogleModels { | |||
| let client = Client::new(); | |||
|
|
|||
| //Send request | |||
| match &self { | |||
There was a problem hiding this comment.
Because the allow-deprecated branches of match need to be separated I would have to repeat the same code twice. Instead I moved it to a reusable method.
There was a problem hiding this comment.
Is that function get_data_studio? If I understand this right, you had to split the studio and vertex calls into their own functions but wanted to re-use certain parts of the code so you had to create Impl GoogleModels to define these implementation specific functions outside of the trait implementations to do so.
Although on second read it looks like get_data_studio is called by get_data and the two separate calls to get_data_studio occur in each deprecated and non-deprecated section. The specific calls for vertex vs studio appears to be because they return different responses
Makes sense and nothing in the code is jumping out as strange.
There was a problem hiding this comment.
In many ways the initial design was just not well organized. Vertex and Studio have different ways of calling the API and processing response. But I had both of those different implementations directly in corresponding branches of the match in the get_data method.
As you observed the non-deprecated and deprecated variants need to be in separate branches so instead of repeating the code I finally cleaned it up and moved to standalone function(s) that can be called from those branches.
No description provided.