Skip to content

Commit

Permalink
feat (ui): introduce message parts for useChat (#4670)
Browse files Browse the repository at this point in the history
  • Loading branch information
@lgrammel
lgrammel authored Feb 5, 2025
1 parent a4a9bc8 commit bcc61d4
Show file tree
Hide file tree
Showing 52 changed files with 5,822 additions and 2,227 deletions.
10 changes: 10 additions & 0 deletions .changeset/fair-mails-pretend.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
'@ai-sdk/ui-utils': patch
'@ai-sdk/svelte': patch
'@ai-sdk/react': patch
'@ai-sdk/solid': patch
'@ai-sdk/vue': patch
'ai': patch
---

feat (ui): introduce message parts for useChat
31 changes: 24 additions & 7 deletions content/docs/04-ai-sdk-ui/02-chatbot.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ export async function POST(req: Request) {
}
```

<Note>
The UI messages have a new `parts` property that contains the message parts.
We recommend rendering the messages using the `parts` property instead of the
`content` property. The parts property supports different message types,
including text, tool invocation, and tool result, and allows for more flexible
and complex chat UIs.
</Note>

In the `Page` component, the `useChat` hook will request to your AI provider endpoint whenever the user submits a message.
The messages are then streamed back in real-time and displayed in the chat UI.

Expand Down Expand Up @@ -510,14 +518,23 @@ export async function POST(req: Request) {
}
```

On the client side, you can access the reasoning text with the `reasoning` property on the message object:
On the client side, you can access the reasoning parts of the message object:

```tsx filename="app/page.tsx" highlight="4"
messages.map(m => (
<div key={m.id}>
{m.role === 'user' ? 'User: ' : 'AI: '}
{m.reasoning && <pre>{m.reasoning}</pre>}
{m.content}
```tsx filename="app/page.tsx"
messages.map(message => (
<div key={message.id} className="whitespace-pre-wrap">
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.parts.map((part, index) => {
// text parts:
if (part.type === 'text') {
return <div key={index}>{part.text}</div>;
}

// reasoning parts:
if (part.type === 'reasoning') {
return <pre key={index}>{part.reasoning}</pre>;
}
})}
</div>
));
```
Expand Down
176 changes: 123 additions & 53 deletions content/docs/04-ai-sdk-ui/03-chatbot-tool-usage.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ The flow is as follows:
1. Client-side tools that should be automatically executed are handled with the `onToolCall` callback.
You can return the tool result from the callback.
1. Client-side tool that require user interactions can be displayed in the UI.
The tool calls and results are available in the `toolInvocations` property of the last assistant message.
The tool calls and results are available as tool invocation parts in the `parts` property of the last assistant message.
1. When the user interaction is done, `addToolResult` can be used to add the tool result to the chat.
1. When there are tool calls in the last assistant message and all tool results are available, the client sends the updated messages back to the server.
This triggers another iteration of this flow.

The tool call and tool executions are integrated into the assistant message as `toolInvocations`.
The tool call and tool executions are integrated into the assistant message as tool invocation parts.
A tool invocation is at first a tool call, and then it becomes a tool result when the tool is executed.
The tool result contains all information about the tool call as well as the result of the tool execution.

Expand Down Expand Up @@ -61,7 +61,7 @@ export async function POST(req: Request) {
const { messages } = await req.json();

const result = streamText({
model: openai('gpt-4-turbo'),
model: openai('gpt-4o'),
messages,
tools: {
// server-side tool with execute function:
Expand Down Expand Up @@ -98,7 +98,8 @@ export async function POST(req: Request) {
### Client-side page

The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming.
Tool invocations are displayed in the chat UI.
Tool invocations are displayed in the chat UI as tool invocation parts.
Please make sure to render the messages using the `parts` property of the message.

There are three things worth mentioning:

Expand All @@ -117,7 +118,7 @@ There are three things worth mentioning:
'use client';

import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';
import { useChat } from 'ai/react';

export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, addToolResult } =
Expand All @@ -140,43 +141,110 @@ export default function Chat() {

return (
<>
{messages?.map((m: Message) => (
<div key={m.id}>
<strong>{m.role}:</strong>
{m.content}
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
const toolCallId = toolInvocation.toolCallId;
const addResult = (result: string) =>
addToolResult({ toolCallId, result });

// render confirmation tool (client-side tool with user interaction)
if (toolInvocation.toolName === 'askForConfirmation') {
return (
<div key={toolCallId}>
{toolInvocation.args.message}
<div>
{'result' in toolInvocation ? (
<b>{toolInvocation.result}</b>
) : (
<>
<button onClick={() => addResult('Yes')}>Yes</button>
<button onClick={() => addResult('No')}>No</button>
</>
)}
</div>
</div>
);
{messages?.map(message => (
<div key={message.id} className="whitespace-pre-wrap">
<strong>{`${message.role}: `}</strong>
{message.parts.map(part => {
switch (part.type) {
// render text parts as simple text:
case 'text':
return part.text;

// for tool invocations, distinguish between the tools and the state:
case 'tool-invocation': {
const callId = part.toolInvocation.toolCallId;

switch (part.toolInvocation.toolName) {
case 'askForConfirmation': {
switch (part.toolInvocation.state) {
case 'call':
return (
<div key={callId} className="text-gray-500">
{part.toolInvocation.args.message}
<div className="flex gap-2">
<button
className="px-4 py-2 font-bold text-white bg-blue-500 rounded hover:bg-blue-700"
onClick={() =>
addToolResult({
toolCallId: callId,
result: 'Yes, confirmed.',
})
}
>
Yes
</button>
<button
className="px-4 py-2 font-bold text-white bg-red-500 rounded hover:bg-red-700"
onClick={() =>
addToolResult({
toolCallId: callId,
result: 'No, denied',
})
}
>
No
</button>
</div>
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Location access allowed:{' '}
{part.toolInvocation.result}
</div>
);
}
break;
}

case 'getLocation': {
switch (part.toolInvocation.state) {
case 'call':
return (
<div key={callId} className="text-gray-500">
Getting location...
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Location: {part.toolInvocation.result}
</div>
);
}
break;
}

case 'getWeatherInformation': {
switch (part.toolInvocation.state) {
// example of pre-rendering streaming tool calls:
case 'partial-call':
return (
<pre key={callId}>
{JSON.stringify(part.toolInvocation, null, 2)}
</pre>
);
case 'call':
return (
<div key={callId} className="text-gray-500">
Getting weather information for{' '}
{part.toolInvocation.args.city}...
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Weather in {part.toolInvocation.args.city}:{' '}
{part.toolInvocation.result}
</div>
);
}
break;
}
}
}
}

// other tools:
return 'result' in toolInvocation ? (
<div key={toolCallId}>
Tool call {`${toolInvocation.toolName}: `}
{toolInvocation.result}
</div>
) : (
<div key={toolCallId}>Calling {toolInvocation.toolName}...</div>
);
})}
<br />
</div>
Expand Down Expand Up @@ -210,24 +278,26 @@ export async function POST(req: Request) {

When the flag is enabled, partial tool calls will be streamed as part of the data stream.
They are available through the `useChat` hook.
The `toolInvocations` property of assistant messages will also contain partial tool calls.
The tool invocation parts of assistant messages will also contain partial tool calls.
You can use the `state` property of the tool invocation to render the correct UI.

```tsx filename='app/page.tsx' highlight="9,10"
export default function Chat() {
// ...
return (
<>
{messages?.map((m: Message) => (
<div key={m.id}>
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
switch (toolInvocation.state) {
case 'partial-call':
return <>render partial tool call</>;
case 'call':
return <>render full tool call</>;
case 'result':
return <>render tool result</>;
{messages?.map(message => (
<div key={message.id}>
{message.parts.map(part => {
if (part.type === 'tool-invocation') {
switch (part.toolInvocation.state) {
case 'partial-call':
return <>render partial tool call</>;
case 'call':
return <>render full tool call</>;
case 'result':
return <>render tool result</>;
}
}
})}
</div>
Expand All @@ -251,7 +321,7 @@ export async function POST(req: Request) {
const { messages } = await req.json();

const result = streamText({
model: openai('gpt-4-turbo'),
model: openai('gpt-4o'),
messages,
tools: {
getWeatherInformation: {
Expand Down
Loading

0 comments on commit bcc61d4

Please sign in to comment.