Add ability to customise model request retry behaviour

[Finndersen]

Currently there appears to be no easy way to customise the Model request retry behaviour, in cases of request error or unexpected response.

In one of my projects I've had to put exception handling around agent.run() to catch request failures (e.g. due to "Model is overloaded" errors from Gemini), however this is inefficient because it re-tries the entire Agent workflow (which may involve multiple model requests).

It would be good to be able to customise the retry logic for each individual model request (Model.request() call), for both:

• Request errors (due to network issue, rate limiting, authorisation..)
• Invalid/unexpected model response content/format (to be able to prompt model to correct itself part way through a multi-step interaction)

[samuelcolvin]

agreed 👍 .

I will say, you could already do this by implementing your own model which wraps the Gemini model and handles retrying.

You might also be interested in #516.

[Finndersen]

I'm not too familiar with all the possible failure modes of the model provider SDKs or APIs that could potentially be resolved automatically by some kind of handler.

There's obviously network or throttling related issues which could be resolved by retrying the entire request.

Are there any other error scenarios due to invalid input provided to the model/API that a handler could potentially resolve by modifying the request data or something?

I was also thinking about some way to check the model response before returning to the user, and automatically re-prompt the LLM to do something differently if the response isn't as expected. But this could be done outside the agent.run() interaction.