Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat (ui): introduce message parts for useChat #4670

Merged
merged 67 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
c2c499c
1
lgrammel Feb 3, 2025
447ca78
2
lgrammel Feb 3, 2025
0ee042a
tweak
lgrammel Feb 3, 2025
793d89d
fx
lgrammel Feb 3, 2025
63911a4
tests
lgrammel Feb 3, 2025
59df9cc
3
lgrammel Feb 3, 2025
b29d26f
text part
lgrammel Feb 3, 2025
af94c6a
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 3, 2025
c0118be
append to response msgs
lgrammel Feb 3, 2025
de16955
test
lgrammel Feb 3, 2025
94289d7
fix
lgrammel Feb 3, 2025
819e673
update-tool-call-result
lgrammel Feb 3, 2025
0037807
fix
lgrammel Feb 3, 2025
72abc10
svelte
lgrammel Feb 3, 2025
9da69dd
solid
lgrammel Feb 3, 2025
f03ac6d
adjust
lgrammel Feb 3, 2025
0525428
extract
lgrammel Feb 3, 2025
eec6111
solid
lgrammel Feb 3, 2025
5413a32
svekte fille
lgrammel Feb 4, 2025
ca8815d
resubmit react
lgrammel Feb 4, 2025
101f5c9
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 4, 2025
b136b30
svelte
lgrammel Feb 4, 2025
4ffbd80
solid
lgrammel Feb 4, 2025
eb35787
vue
lgrammel Feb 4, 2025
3cb1c37
tool call
lgrammel Feb 4, 2025
be86bce
deprecate
lgrammel Feb 4, 2025
980083d
x
lgrammel Feb 4, 2025
ddec817
revert
lgrammel Feb 4, 2025
d1a7e9f
fix types
lgrammel Feb 4, 2025
f2d9816
1
lgrammel Feb 4, 2025
5c2793e
start new reasoning / text parts
lgrammel Feb 4, 2025
91f12f7
renane
lgrammel Feb 4, 2025
bc26917
snapshot
lgrammel Feb 4, 2025
60280d7
structure
lgrammel Feb 4, 2025
4211fc0
1
lgrammel Feb 4, 2025
4cf4c2b
2
lgrammel Feb 4, 2025
58918cd
3
lgrammel Feb 4, 2025
bc54f81
4
lgrammel Feb 4, 2025
1bc6710
remove
lgrammel Feb 4, 2025
2f8e410
5
lgrammel Feb 4, 2025
43da544
anthropic
lgrammel Feb 4, 2025
7070873
vue
lgrammel Feb 4, 2025
586a6f3
parts
lgrammel Feb 4, 2025
bd64a3e
ffs
lgrammel Feb 4, 2025
d3b0d25
stylor
lgrammel Feb 4, 2025
da51bab
revert deprecation
lgrammel Feb 4, 2025
1e02eaf
fx
lgrammel Feb 4, 2025
e981e65
fx
lgrammel Feb 4, 2025
adf98f0
asdasd
lgrammel Feb 4, 2025
ab0e959
send
lgrammel Feb 4, 2025
fb5f2f0
ok
lgrammel Feb 4, 2025
9d39748
clean
lgrammel Feb 4, 2025
2bbaa51
change order
lgrammel Feb 4, 2025
807b645
solid
lgrammel Feb 4, 2025
84df335
clean
lgrammel Feb 4, 2025
7e4ba9b
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 4, 2025
8c8c9e3
svelte
lgrammel Feb 4, 2025
dd92ee7
jsdoc
lgrammel Feb 4, 2025
f4af54c
update reasoning example
lgrammel Feb 4, 2025
5bb4c47
part index
lgrammel Feb 4, 2025
78268a6
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 4, 2025
e2bffd1
@internal
lgrammel Feb 4, 2025
68c76fe
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 5, 2025
be6c4fc
guide 1
lgrammel Feb 5, 2025
dd1a4d5
guide
lgrammel Feb 5, 2025
6758d3d
guide
lgrammel Feb 5, 2025
4edca9d
Merge branch 'main' into lg/Pcy8Aanj
lgrammel Feb 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .changeset/fair-mails-pretend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
'@ai-sdk/ui-utils': patch
'@ai-sdk/svelte': patch
'@ai-sdk/react': patch
'@ai-sdk/solid': patch
'@ai-sdk/vue': patch
'ai': patch
---

feat (ui): introduce message parts for useChat
31 changes: 24 additions & 7 deletions content/docs/04-ai-sdk-ui/02-chatbot.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,14 @@ export async function POST(req: Request) {
}
```

<Note>
The UI messages have a new `parts` property that contains the message parts.
We recommend rendering the messages using the `parts` property instead of the
`content` property. The parts property supports different message types,
including text, tool invocation, and tool result, and allows for more flexible
and complex chat UIs.
</Note>

In the `Page` component, the `useChat` hook will request to your AI provider endpoint whenever the user submits a message.
The messages are then streamed back in real-time and displayed in the chat UI.

Expand Down Expand Up @@ -510,14 +518,23 @@ export async function POST(req: Request) {
}
```

On the client side, you can access the reasoning text with the `reasoning` property on the message object:
On the client side, you can access the reasoning parts of the message object:

```tsx filename="app/page.tsx" highlight="4"
messages.map(m => (
<div key={m.id}>
{m.role === 'user' ? 'User: ' : 'AI: '}
{m.reasoning && <pre>{m.reasoning}</pre>}
{m.content}
```tsx filename="app/page.tsx"
messages.map(message => (
<div key={message.id} className="whitespace-pre-wrap">
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.parts.map((part, index) => {
// text parts:
if (part.type === 'text') {
return <div key={index}>{part.text}</div>;
}

// reasoning parts:
if (part.type === 'reasoning') {
return <pre key={index}>{part.reasoning}</pre>;
}
})}
</div>
));
```
Expand Down
176 changes: 123 additions & 53 deletions content/docs/04-ai-sdk-ui/03-chatbot-tool-usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ The flow is as follows:
1. Client-side tools that should be automatically executed are handled with the `onToolCall` callback.
You can return the tool result from the callback.
1. Client-side tool that require user interactions can be displayed in the UI.
The tool calls and results are available in the `toolInvocations` property of the last assistant message.
The tool calls and results are available as tool invocation parts in the `parts` property of the last assistant message.
1. When the user interaction is done, `addToolResult` can be used to add the tool result to the chat.
1. When there are tool calls in the last assistant message and all tool results are available, the client sends the updated messages back to the server.
This triggers another iteration of this flow.

The tool call and tool executions are integrated into the assistant message as `toolInvocations`.
The tool call and tool executions are integrated into the assistant message as tool invocation parts.
A tool invocation is at first a tool call, and then it becomes a tool result when the tool is executed.
The tool result contains all information about the tool call as well as the result of the tool execution.

Expand Down Expand Up @@ -61,7 +61,7 @@ export async function POST(req: Request) {
const { messages } = await req.json();

const result = streamText({
model: openai('gpt-4-turbo'),
model: openai('gpt-4o'),
messages,
tools: {
// server-side tool with execute function:
Expand Down Expand Up @@ -98,7 +98,8 @@ export async function POST(req: Request) {
### Client-side page

The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming.
Tool invocations are displayed in the chat UI.
Tool invocations are displayed in the chat UI as tool invocation parts.
Please make sure to render the messages using the `parts` property of the message.

There are three things worth mentioning:

Expand All @@ -117,7 +118,7 @@ There are three things worth mentioning:
'use client';

import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';
import { useChat } from 'ai/react';

export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, addToolResult } =
Expand All @@ -140,43 +141,110 @@ export default function Chat() {

return (
<>
{messages?.map((m: Message) => (
<div key={m.id}>
<strong>{m.role}:</strong>
{m.content}
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
const toolCallId = toolInvocation.toolCallId;
const addResult = (result: string) =>
addToolResult({ toolCallId, result });

// render confirmation tool (client-side tool with user interaction)
if (toolInvocation.toolName === 'askForConfirmation') {
return (
<div key={toolCallId}>
{toolInvocation.args.message}
<div>
{'result' in toolInvocation ? (
<b>{toolInvocation.result}</b>
) : (
<>
<button onClick={() => addResult('Yes')}>Yes</button>
<button onClick={() => addResult('No')}>No</button>
</>
)}
</div>
</div>
);
{messages?.map(message => (
<div key={message.id} className="whitespace-pre-wrap">
<strong>{`${message.role}: `}</strong>
{message.parts.map(part => {
switch (part.type) {
// render text parts as simple text:
case 'text':
return part.text;

// for tool invocations, distinguish between the tools and the state:
case 'tool-invocation': {
const callId = part.toolInvocation.toolCallId;

switch (part.toolInvocation.toolName) {
case 'askForConfirmation': {
switch (part.toolInvocation.state) {
case 'call':
return (
<div key={callId} className="text-gray-500">
{part.toolInvocation.args.message}
<div className="flex gap-2">
<button
className="px-4 py-2 font-bold text-white bg-blue-500 rounded hover:bg-blue-700"
onClick={() =>
addToolResult({
toolCallId: callId,
result: 'Yes, confirmed.',
})
}
>
Yes
</button>
<button
className="px-4 py-2 font-bold text-white bg-red-500 rounded hover:bg-red-700"
onClick={() =>
addToolResult({
toolCallId: callId,
result: 'No, denied',
})
}
>
No
</button>
</div>
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Location access allowed:{' '}
{part.toolInvocation.result}
</div>
);
}
break;
}

case 'getLocation': {
switch (part.toolInvocation.state) {
case 'call':
return (
<div key={callId} className="text-gray-500">
Getting location...
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Location: {part.toolInvocation.result}
</div>
);
}
break;
}

case 'getWeatherInformation': {
switch (part.toolInvocation.state) {
// example of pre-rendering streaming tool calls:
case 'partial-call':
return (
<pre key={callId}>
{JSON.stringify(part.toolInvocation, null, 2)}
</pre>
);
case 'call':
return (
<div key={callId} className="text-gray-500">
Getting weather information for{' '}
{part.toolInvocation.args.city}...
</div>
);
case 'result':
return (
<div key={callId} className="text-gray-500">
Weather in {part.toolInvocation.args.city}:{' '}
{part.toolInvocation.result}
</div>
);
}
break;
}
}
}
}

// other tools:
return 'result' in toolInvocation ? (
<div key={toolCallId}>
Tool call {`${toolInvocation.toolName}: `}
{toolInvocation.result}
</div>
) : (
<div key={toolCallId}>Calling {toolInvocation.toolName}...</div>
);
})}
<br />
</div>
Expand Down Expand Up @@ -210,24 +278,26 @@ export async function POST(req: Request) {

When the flag is enabled, partial tool calls will be streamed as part of the data stream.
They are available through the `useChat` hook.
The `toolInvocations` property of assistant messages will also contain partial tool calls.
The tool invocation parts of assistant messages will also contain partial tool calls.
You can use the `state` property of the tool invocation to render the correct UI.

```tsx filename='app/page.tsx' highlight="9,10"
export default function Chat() {
// ...
return (
<>
{messages?.map((m: Message) => (
<div key={m.id}>
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
switch (toolInvocation.state) {
case 'partial-call':
return <>render partial tool call</>;
case 'call':
return <>render full tool call</>;
case 'result':
return <>render tool result</>;
{messages?.map(message => (
<div key={message.id}>
{message.parts.map(part => {
if (part.type === 'tool-invocation') {
switch (part.toolInvocation.state) {
case 'partial-call':
return <>render partial tool call</>;
case 'call':
return <>render full tool call</>;
case 'result':
return <>render tool result</>;
}
}
})}
</div>
Expand All @@ -251,7 +321,7 @@ export async function POST(req: Request) {
const { messages } = await req.json();

const result = streamText({
model: openai('gpt-4-turbo'),
model: openai('gpt-4o'),
messages,
tools: {
getWeatherInformation: {
Expand Down
Loading
Loading