Anthropic sometimes fails with parallel tool calls

[sydney-runkle]

Initially reported in #653. Needs more info, cc @ioga - a MRE would be great.

Happy to open a PR with a fix once we can reproduce the issue + guarantee that the solution doesn't change message order in responses.

[ioga]

@sydney-runkle here you go:

import logging
from pydantic_ai import Agent

logging.basicConfig(
    format="%(asctime)s | %(levelname)8s | %(filename)s:%(lineno)d | %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
    level=logging.WARNING,
)

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

async def rel_tool(name: str) -> str:
    """Get the knowledge about the given entity."""
    logger.debug("Getting knowledge about {}.".format(name))
    data = {
        "alice": "alice is bob's wife",
        "bob": "bob is alice's husband",
        "charlie": "charlie is alice's son",
        "daisy": "daisy is bob's daughter and charlie's younger sister",
    }
    return data[name.lower()]


SYSTEM_PROMPT = """
Use the `rel_tool` to get information about a specific person.
If you need to use `rel_tool` to get information about multiple people, try
to call them in parallel as much as possible.
Think step by step and then provide a single most probably consise answer.
"""

def main(input: str = None):
    agent = Agent(
        "claude-3-5-sonnet-latest",
        system_prompt=SYSTEM_PROMPT,
        tools=[rel_tool],
    )

    if input is None:
        input = "Alice, Bob, Charlie and Daisy are a family. Who is the youngest?"
    result = agent.run_sync(input)
    print(result.data)

if __name__ == "__main__":
    main()

on 0.0.19, I get an error with this code:

anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': "messages.2: the following `tool_use` ids were not found in `tool_result` blocks: {'toolu_0186A8Pb8m12VtL2ttnH7vBG', 'toolu_015rX8DRRVAeGXkbq57gutCV', 'toolu_01F26ERdRD4VNnY8UheCefTv'}"}}

and with my patch, it works.

[sydney-runkle]

Side note, I've just added parallel_tool_calls as a ModelSettings attribute, if you want to use that to disable parallel tool calls in the meantime. I'll work to repro and troubleshoot this specific issue tomorrow.

[ioga]

I've observed in our use case that if you don't ask claude to do parallel tool calls, it'd hardly ever do them, which probably the reason this bug doesn't surface often by default.

Without the parallel tool calls, our anthropic-based agent is several times slower than a gpt-4-based one. This makes any UX much worse, even if the quality is better.

[sydney-runkle]

I'm able to repro this - thanks for the example!

[dmyung]

Just wanted to chime/upvote in to say I'm running into this issue quite a bit after having on a multi-agent and multi-tool setup. The 400 error pops up with much higher frequency (now that i understand that it was likely a parallelism issue with the plurality of agents I've been adding)

Running anthropic models on Bedrock, looking forward to the resolution!