return_direct in Tool Decorator has no effect

[koenigfb]

Version:
langgraph 0.2.61
langchain-core: 0.3.29

Problem:
I have a chat assistent which may call a tool that returns data as json which should be returned back to the user. Therefore the graph should end after the tool call.

However setting @tool(return_direct=True) to true or false has no effect. I also tried StructuredTool.from_function(func=get_report, return_direct=True).

The message flow is exactly the same.

Here is the code:

@tool("report", return_direct=True)
def create_report(query: str = None, **kwargs) -> str:
    """Extracts report information from the user's query."""
    try:
        return json.dumps({
                    "event": "report",
                    "data": {
                        "type": "report",
                        "data": {
                            "date": "12.12.2014",
                            "id": "2",
                            "grades": ["a", "b"],
                        }
                    }
                })

available_functions = {
    "address": get_address,
    "report": report,
}

----------------------------------------------


tools = [tool for _, tool in available_functions.items()]
llm = ChatOpenAI(model=model, streaming=False, callbacks=[langfuse_handler])
llm_with_tools = llm.bind_tools(tools)

def create_tool_node_with_fallback(tools: list) -> dict:
    return ToolNode(tools).with_fallbacks(
        [RunnableLambda(handle_tool_error)], exception_key="error"
    )

### State
class State(TypedDict):
    """
    Represents the state of our graph.

    Attributes:
        messages: List of chat messages.
    """
    messages: Annotated[list[AnyMessage], add_messages]

class Assistant:
    def __init__(self, runnable: Runnable):
        self.runnable = runnable

    def __call__(self, state: State, config: RunnableConfig):
        try:
            result = self.runnable.invoke(state, config)
            if not result.content and not result.tool_calls:
                messages = state["messages"] 
                state = {**state, "messages": messages}
                result = self.runnable.invoke(state, config)
            return {"messages": result}
        except Exception as e:
            logger.error(f"Error in Assistant: {e}")

def get_runnable():
    builder = StateGraph(State)
    
    configured_assistant = Assistant(
        assistant_runnable.with_config({
            "stream_events": True,
        })
    )
    
    builder.add_node("assistant", configured_assistant)
    builder.add_node("tools", create_tool_node_with_fallback(tools))
    builder.add_edge(START, "assistant")
    builder.add_conditional_edges(
        "assistant",
        tools_condition,
    )
    builder.add_edge("tools", "assistant")

    memory = MemorySaver()
    app = builder.compile(
        checkpointer=memory
    )
    return app

Is there anything missing? Or is it a bug?

Thx for the help!

[hinthornw]

In your graph, you've written a direct edge builder.add_edge("tools", "assistant"). This means "always return to the assistant after the tools node is called." If you're writing your own graph and not using a higher-level prebuilt abstraction, then you'll also have to include the logic on whether or not to return directly yourself. In this case you'd replace that with a conditional edge and then check to see whther the tool message is from a tool you've marked with `return_direct.

return_direct is designed for and correctly interpreted by higher level abstractions like create_react_agent or LangChain's legacy AgentExector.

[vbarda]

see this for reference https://github.com/langchain-ai/langgraph/blob/main/libs/langgraph/langgraph/prebuilt/chat_agent_executor.py#L688

[koenigfb]

thank you for clarifying!

I also managed to achieve this by checking the metadata for the decorator values:

def should_return_direct(state: GraphState) -> Literal["__end__", "agent"]:
    """
    Determine whether to continue or end the workflow based on tool return_direct parameter.
    For return_direct=True tools, ensure their output is included in the final response.
    """
     last_metadata = state.get('metadata', [])

    if not last_metadata:
        return END

    needs_agent = any(not tool_meta['return_direct'] for tool_meta in last_metadata)
    
    return "agent" if needs_agent else END