[Bug]: Vertex AI image generation response does not contain 'predictions'

[ushuz]

What happened?

We frequently encounter the following error when using imagen-3 on Vertex AI, which indicates some kinds of error handling issue with LiteLLM:

litellm.exceptions.InternalServerError: litellm.InternalServerError: image generation response does not contain 'predictions', got {}

Relevant log output

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2966, in async_function_with_fallbacks
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2827, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3198, in async_function_with_retries
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3093, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3207, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 1701, in _aimage_generation
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 1683, in _aimage_generation
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1300, in wrapper_async
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1159, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 4344, in aimage_generation
    raise exception_type(
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 4337, in aimage_generation
    response = await init_response  # type: ignore
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/vertex_ai/image_generation/image_generation_handler.py", line 227, in aimage_generation
    return self.process_image_generation_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/vertex_ai/image_generation/image_generation_handler.py", line 25, in process_image_generation_response
    raise litellm.InternalServerError(
litellm.exceptions.InternalServerError: litellm.InternalServerError: image generation response does not contain 'predictions', got {}No fallback model group found for original model_group=imagen-3. Fallbacks=[...] LiteLLM Retried: 1 times, LiteLLM Max Retries: 2

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.59.6

Twitter / LinkedIn details

No response