Google suts

[bkorycki]

No description provided.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[wpietri]

I am having trouble running a benchmark with this. I get:

Traceback (most recent call last):
  File "/home/william/projects/mlcommons/modelbench/src/modelbench/benchmark_runner.py", line 330, in handle_item
    raw_response = mg_sut.evaluate(raw_request)
  File "/home/william/projects/mlcommons/modelbench/plugins/google/modelgauge/suts/google_genai_client.py", line 81, in evaluate
    return GoogleGenAiResponse(**response.to_dict())
  File "/home/william/.cache/pypoetry/virtualenvs/modelbench-NLtplA-d-py3.10/lib/python3.10/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for GoogleGenAiResponse
candidates.0.content
  Field required [type=missing, input_value={'finish_reason': 3, 'ind...[], 'avg_logprobs': 0.0}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing