[UnprocessableEntityError]: status_code: 422, body: data=None message='No valid tool call or response generated'

[Zephyruss1]

cohere-ai/notebooks#237

[ai-yann]

Issue Type

Bug Report

Description

Getting UnprocessableEntityError when attempting to use Cohere's command-r-plus model with LangChain's with_structured_output functionality. The error indicates no valid tool call or response was generated when trying to extract a structured response matching a Pydantic schema.

Error message: UnprocessableEntityError: status_code: 422, body: data=None message='No valid tool call or response generated'

Steps to Reproduce

1. Install required packages:

   pip install pydantic==2.9.2 langchain-cohere==0.3.0 python-dotenv==1.0.1

2. Set up COHERE_API_KEY environment variable

3. Create a Pydantic model for the structured output:

   from pydantic import BaseModel, Field, List
   
   class Plan(BaseModel):
       """Plan to follow in future"""
       steps: List[str] = Field(
           description="different steps to follow, should be in sorted order"
       )

4. Set up the LangChain chain with Cohere:

   from langchain_core.prompts import ChatPromptTemplate
   from langchain_cohere import ChatCohere
   
   planner_prompt = ChatPromptTemplate.from_messages([
       (
           "system",
           """For the given objective, come up with a simple step by step plan. \
   This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \
   The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps.""",
       ),
       ("placeholder", "{messages}"),
   ])
   
   planner = planner_prompt | ChatCohere(
       model="command-r-plus"
   ).with_structured_output(Plan)

5. Try to invoke the chain:

   planner.invoke({
       "messages": [
           ("user", "what is the hometown of the current Australia open winner?")
       ]
   })

Expected Behavior

The chain should:

1. Process the input question
2. Generate a structured response matching the Plan schema
3. Return a list of steps to answer the given question

System Information

• Operating System: Linux
• Python version: 3.12
• Package versions:
  • pydantic: 2.9.2
  • langchain-cohere: 0.3.0
  • python-dotenv: 1.0.1
• Cohere API version: Using command-r-plus model

Error Output

UnprocessableEntityError                  Traceback (most recent call last)
Cell In[25], line 1
----> 1 planner.invoke(
      2     {
      3         "messages": [
      4             ("user", "what is the hometown of the current Australia open winner?")
      5         ]
      6     }
      7 )

[... truncated for brevity ...]

UnprocessableEntityError: status_code: 422, body: data=None message='No valid tool call or response generated'

Additional Context

• Using the latest version of langchain-cohere (0.3.0)
• The error seems to originate from the Cohere API's response processing
• The same prompt structure works with other LLM providers

[ai-yann]

I was able to reproduce your issue once I modified the List to use the built-in list in Python:

STEPS I TOOK TO REPRODUCE YOUR ISSUE

requirements.txt

pydantic==2.9.2
langchain-cohere==0.3.0
python-dotenv==1.0.1

cohere_repro.py

from pydantic import BaseModel, Field
from langchain_core.prompts import ChatPromptTemplate
from langchain_cohere import ChatCohere

# Create Pydantic model for structured output
class Plan(BaseModel):
    """Plan to follow in future"""
    steps: list[str] = Field(
        description="different steps to follow, should be in sorted order"
    )

# Set up the LangChain chain with Cohere
planner_prompt = ChatPromptTemplate.from_messages([
    (
        "system",
        """For the given objective, come up with a simple step by step plan. \
This plan should involve individual tasks, that if executed correctly will yield the correct answer. Do not add any superfluous steps. \
The result of the final step should be the final answer. Make sure that each step has all the information needed - do not skip steps.""",
    ),
    ("placeholder", "{messages}"),
])
planner = planner_prompt | ChatCohere(
    model="command-r-plus"
).with_structured_output(Plan)

# Try to invoke the chain
planner.invoke({
    "messages": [
        ("user", "what is the hometown of the current Australia open winner?")
    ]
})