diff --git a/.changeset/wicked-yaks-reply.md b/.changeset/wicked-yaks-reply.md new file mode 100644 index 000000000000..8bd3c0e99a91 --- /dev/null +++ b/.changeset/wicked-yaks-reply.md @@ -0,0 +1,5 @@ +--- +'@ai-sdk/amazon-bedrock': major +--- + +feat (provider/amazon-bedrock): remove dependence on AWS SDK Bedrock client library diff --git a/content/providers/01-ai-sdk-providers/08-amazon-bedrock.mdx b/content/providers/01-ai-sdk-providers/08-amazon-bedrock.mdx index d1b376c01e43..1def93d55d45 100644 --- a/content/providers/01-ai-sdk-providers/08-amazon-bedrock.mdx +++ b/content/providers/01-ai-sdk-providers/08-amazon-bedrock.mdx @@ -90,26 +90,15 @@ const bedrock = createAmazonBedrock({ secretAccessKey: 'xxxxxxxxx', sessionToken: 'xxxxxxxxx', }); - -// or with bedrockOptions -const bedrock = createAmazonBedrock({ - bedrockOptions: { - region: 'us-east-1', - credentials: { - // ... - }, - }, -}); ``` - The top level credentials settings below fall back to environment variable - defaults. These may be set by your serverless environment without your - awareness, which can lead to merged/conflicting credential values and provider - errors around failed authentication. If you're experiencing issues try (1) - using the `bedrockOptions` object as it will take precedence over the other - settings and does not inherit environment variable values, or (2) explicitly - specifying all settings (even if `undefined`) to avoid any defaults. + The credentials settings fall back to environment variable defaults described + below. These may be set by your serverless environment without your awareness, + which can lead to merged/conflicting credential values and provider errors + around failed authentication. If you're experiencing issues be sure you are + explicitly specifying all settings (even if `undefined`) to avoid any + defaults. You can use the following optional settings to customize the Amazon Bedrock provider instance: @@ -134,19 +123,6 @@ You can use the following optional settings to customize the Amazon Bedrock prov Optional. The AWS session token that you want to use for the API calls. It uses the `AWS_SESSION_TOKEN` environment variable by default. -- **bedrockOptions** _object_ - - Optional. The configuration options used by the [Amazon Bedrock Library](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-bedrock-runtime/) - (`BedrockRuntimeClientConfig`), including: - - - **region** _string_ - The AWS region that you want to use for the API calls. - - - **credentials** _object_ - The AWS credentials that you want to use for the API calls. - - When `bedrockOptions` are provided, the `region`, `accessKeyId`, and `secretAccessKey` settings are ignored. - ## Language Models You can create models that call the Bedrock API using the provider instance. @@ -316,3 +292,76 @@ The following optional settings are available for Bedrock Titan embedding models | ------------------------------ | ------------------ | ------------------- | | `amazon.titan-embed-text-v1` | 1536 | | | `amazon.titan-embed-text-v2:0` | 1024 | | + +## Response Headers + +The Amazon Bedrock provider will return the response headers associated with +network requests made of the Bedrock servers. + +```ts +import { bedrock } from '@ai-sdk/amazon-bedrock'; +import { generateText } from 'ai'; + +const { text } = await generateText({ + model: bedrock('meta.llama3-70b-instruct-v1:0'), + prompt: 'Write a vegetarian lasagna recipe for 4 people.', +}); + +console.log(result.response.headers); +``` + +Below is sample output where you can see the `x-amzn-requestid` header. This can +be useful for correlating Bedrock API calls with requests made by the AI SDK: + +```js highlight="6" +{ + connection: 'keep-alive', + 'content-length': '2399', + 'content-type': 'application/json', + date: 'Fri, 07 Feb 2025 04:28:30 GMT', + 'x-amzn-requestid': 'c9f3ace4-dd5d-49e5-9807-39aedfa47c8e' +} +``` + +This information is also available with `streamText`: + +```ts +import { bedrock } from '@ai-sdk/amazon-bedrock'; +import { streamText } from 'ai'; + +const result = streamText({ + model: bedrock('meta.llama3-70b-instruct-v1:0'), + prompt: 'Write a vegetarian lasagna recipe for 4 people.', +}); +for await (const textPart of result.textStream) { + process.stdout.write(textPart); +} +console.log('Response headers:', (await result.response).headers); +``` + +With sample output as: + +```js highlight="6" +{ + connection: 'keep-alive', + 'content-type': 'application/vnd.amazon.eventstream', + date: 'Fri, 07 Feb 2025 04:33:37 GMT', + 'transfer-encoding': 'chunked', + 'x-amzn-requestid': 'a976e3fc-0e45-4241-9954-b9bdd80ab407' +} +``` + +## Migrating from `@ai-sdk/amazon-bedrock` pre-v2.x + +The Amazon Bedrock provider was rewritten in version 2.x to remove the +dependency on the `@aws-sdk/client-bedrock-runtime` package. + +The `bedrockOptions` provider setting previously available has been removed. If +you were using the `bedrockOptions` object, you should now use the `region`, +`accessKeyId`, `secretAccessKey`, and `sessionToken` settings directly instead. + +Note that you may need to set all of these explicitly, e.g. even if you're not +using `sessionToken`, set it to `undefined`. If you're running in a serverless +environment, there may be default environment variables set by your containing +environment that the Amazon Bedrock provider will then pick up and could +conflict with the ones you're intending to use. diff --git a/examples/ai-core/.env.example b/examples/ai-core/.env.example index e86e7914cf75..a5333d2e8350 100644 --- a/examples/ai-core/.env.example +++ b/examples/ai-core/.env.example @@ -1,5 +1,6 @@ ANTHROPIC_API_KEY="" AWS_ACCESS_KEY_ID="" +AWS_ACCOUNT_ID="" AWS_SECRET_ACCESS_KEY="" AWS_REGION="" AZURE_API_KEY="" diff --git a/examples/ai-core/src/generate-object/amazon-bedrock.ts b/examples/ai-core/src/generate-object/amazon-bedrock.ts index 5b120dd092bd..a306fdf638bb 100644 --- a/examples/ai-core/src/generate-object/amazon-bedrock.ts +++ b/examples/ai-core/src/generate-object/amazon-bedrock.ts @@ -5,7 +5,9 @@ import { z } from 'zod'; async function main() { const result = await generateObject({ - model: bedrock('anthropic.claude-3-5-sonnet-20240620-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), schema: z.object({ recipe: z.object({ name: z.string(), diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-chatbot.ts b/examples/ai-core/src/generate-text/amazon-bedrock-chatbot.ts index c4b354e95fab..d089e21b7cab 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-chatbot.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-chatbot.ts @@ -21,7 +21,9 @@ async function main() { } const { text, toolCalls, toolResults, response } = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), tools: { weatherTool }, system: `You are a helpful, respectful and honest assistant. If the weather is requested use the `, messages, diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-guardrails.ts b/examples/ai-core/src/generate-text/amazon-bedrock-guardrails.ts index e8fb9d3db7e1..51428d22a37f 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-guardrails.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-guardrails.ts @@ -4,7 +4,9 @@ import 'dotenv/config'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), prompt: 'Invent a new fake holiday and describe its traditions. ' + 'You are a comedian and should insult the audience as much as possible.', diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-image-url.ts b/examples/ai-core/src/generate-text/amazon-bedrock-image-url.ts index 2762c217e81b..e1872fe2a515 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-image-url.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-image-url.ts @@ -4,7 +4,9 @@ import 'dotenv/config'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), maxTokens: 512, messages: [ { diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-image.ts b/examples/ai-core/src/generate-text/amazon-bedrock-image.ts index f91033061ee0..1034b623b867 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-image.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-image.ts @@ -5,7 +5,9 @@ import fs from 'node:fs'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), maxTokens: 512, messages: [ { diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-prefilled-assistant-message.ts b/examples/ai-core/src/generate-text/amazon-bedrock-prefilled-assistant-message.ts index 1d15221d47aa..cc6ceeab2d76 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-prefilled-assistant-message.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-prefilled-assistant-message.ts @@ -4,7 +4,9 @@ import 'dotenv/config'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), messages: [ { role: 'user', diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-tool-call.ts b/examples/ai-core/src/generate-text/amazon-bedrock-tool-call.ts index 25dd0ba2d070..e1a11fb463f9 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-tool-call.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-tool-call.ts @@ -6,7 +6,9 @@ import { bedrock } from '@ai-sdk/amazon-bedrock'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-5-sonnet-20240620-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), tools: { weather: weatherTool, cityAttractions: tool({ diff --git a/examples/ai-core/src/generate-text/amazon-bedrock-tool-choice.ts b/examples/ai-core/src/generate-text/amazon-bedrock-tool-choice.ts index a73e7216b945..1cf53866c322 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock-tool-choice.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock-tool-choice.ts @@ -6,7 +6,9 @@ import { bedrock } from '@ai-sdk/amazon-bedrock'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), maxTokens: 512, tools: { weather: weatherTool, diff --git a/examples/ai-core/src/generate-text/amazon-bedrock.ts b/examples/ai-core/src/generate-text/amazon-bedrock.ts index 183ba70cc916..32559ed65961 100644 --- a/examples/ai-core/src/generate-text/amazon-bedrock.ts +++ b/examples/ai-core/src/generate-text/amazon-bedrock.ts @@ -4,14 +4,17 @@ import 'dotenv/config'; async function main() { const result = await generateText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), - prompt: 'Invent a new holiday and describe its traditions.', + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), + prompt: 'Give me an overview of the New Zealand Fiordland National Park.', }); console.log(result.text); console.log(); console.log('Token usage:', result.usage); console.log('Finish reason:', result.finishReason); + console.log('Response headers:', result.response.headers); } main().catch(console.error); diff --git a/examples/ai-core/src/stream-object/amazon-bedrock.ts b/examples/ai-core/src/stream-object/amazon-bedrock.ts index 9f8466ed52d5..3032a0c49432 100644 --- a/examples/ai-core/src/stream-object/amazon-bedrock.ts +++ b/examples/ai-core/src/stream-object/amazon-bedrock.ts @@ -5,7 +5,9 @@ import { z } from 'zod'; async function main() { const result = streamObject({ - model: bedrock('anthropic.claude-3-5-sonnet-20240620-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), schema: z.object({ characters: z.array( z.object({ diff --git a/examples/ai-core/src/stream-text/amazon-bedrock-chatbot.ts b/examples/ai-core/src/stream-text/amazon-bedrock-chatbot.ts index c0c159ee4112..286e6658bd61 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock-chatbot.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock-chatbot.ts @@ -18,7 +18,9 @@ async function main() { messages.push({ role: 'user', content: userInput }); const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), tools: { weather: tool({ description: 'Get the weather in a location', diff --git a/examples/ai-core/src/stream-text/amazon-bedrock-fullstream.ts b/examples/ai-core/src/stream-text/amazon-bedrock-fullstream.ts index 5316abadac67..fc1c418e5125 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock-fullstream.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock-fullstream.ts @@ -6,7 +6,9 @@ import { weatherTool } from '../tools/weather-tool'; async function main() { const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), tools: { weather: weatherTool, cityAttractions: { diff --git a/examples/ai-core/src/stream-text/amazon-bedrock-image.ts b/examples/ai-core/src/stream-text/amazon-bedrock-image.ts index 6375b2b9f923..f8b1bd6d5f61 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock-image.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock-image.ts @@ -5,7 +5,9 @@ import fs from 'node:fs'; async function main() { const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), maxTokens: 512, messages: [ { diff --git a/examples/ai-core/src/stream-text/amazon-bedrock-multi-step-continue.ts b/examples/ai-core/src/stream-text/amazon-bedrock-multi-step-continue.ts index 280007001f2b..db7af42bf8c0 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock-multi-step-continue.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock-multi-step-continue.ts @@ -4,7 +4,9 @@ import 'dotenv/config'; async function main() { const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), maxTokens: 512, // artificial limit for demo purposes maxSteps: 5, experimental_continueSteps: true, diff --git a/examples/ai-core/src/stream-text/amazon-bedrock-pdf.ts b/examples/ai-core/src/stream-text/amazon-bedrock-pdf.ts index e1409b62574e..001a4d22e670 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock-pdf.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock-pdf.ts @@ -5,7 +5,9 @@ import fs from 'node:fs'; async function main() { const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), messages: [ { role: 'user', diff --git a/examples/ai-core/src/stream-text/amazon-bedrock.ts b/examples/ai-core/src/stream-text/amazon-bedrock.ts index ced2bb652b44..78e88d6aadb3 100644 --- a/examples/ai-core/src/stream-text/amazon-bedrock.ts +++ b/examples/ai-core/src/stream-text/amazon-bedrock.ts @@ -4,8 +4,10 @@ import 'dotenv/config'; async function main() { const result = streamText({ - model: bedrock('anthropic.claude-3-haiku-20240307-v1:0'), - prompt: 'Invent a new holiday and describe its traditions.', + model: bedrock( + `arn:aws:bedrock:${process.env.AWS_REGION}:${process.env.AWS_ACCOUNT_ID}:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0`, + ), + prompt: 'Give me an overview of the New Zealand Fiordland National Park.', }); for await (const textPart of result.textStream) { @@ -15,6 +17,7 @@ async function main() { console.log(); console.log('Token usage:', await result.usage); console.log('Finish reason:', await result.finishReason); + console.log('Response headers:', (await result.response).headers); } main().catch(console.error); diff --git a/packages/amazon-bedrock/package.json b/packages/amazon-bedrock/package.json index 7007fab674a0..9f13d2068edd 100644 --- a/packages/amazon-bedrock/package.json +++ b/packages/amazon-bedrock/package.json @@ -32,13 +32,13 @@ "dependencies": { "@ai-sdk/provider": "1.0.7", "@ai-sdk/provider-utils": "2.1.6", - "@aws-sdk/client-bedrock-runtime": "^3.663.0" + "@smithy/eventstream-codec": "^4.0.1", + "@smithy/util-utf8": "^4.0.0", + "aws4fetch": "^1.0.20" }, "devDependencies": { - "@smithy/types": "^3.5.0", "@types/node": "^18.19.54", "@vercel/ai-tsconfig": "workspace:*", - "aws-sdk-client-mock": "^4.0.2", "tsup": "^8.3.0", "typescript": "5.6.3", "zod": "3.23.8" diff --git a/packages/amazon-bedrock/src/bedrock-api-types.ts b/packages/amazon-bedrock/src/bedrock-api-types.ts new file mode 100644 index 000000000000..a4cdbc15b67e --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-api-types.ts @@ -0,0 +1,122 @@ +import { Resolvable } from '@ai-sdk/provider-utils'; + +export interface BedrockConverseInput { + system?: Array<{ text: string }>; + messages: Array<{ + role: string; + content: Array; + }>; + toolConfig?: BedrockToolConfiguration; + inferenceConfig?: { + maxTokens?: number; + temperature?: number; + topP?: number; + stopSequences?: string[]; + }; + additionalModelRequestFields?: Record; + guardrailConfig?: + | BedrockGuardrailConfiguration + | BedrockGuardrailStreamConfiguration + | undefined; +} + +export interface BedrockGuardrailConfiguration { + guardrails?: Array<{ + name: string; + description?: string; + parameters?: Record; + }>; +} + +export type BedrockGuardrailStreamConfiguration = BedrockGuardrailConfiguration; + +export interface BedrockToolInputSchema { + json: Record; +} + +export interface BedrockTool { + toolSpec: { + name: string; + description?: string; + inputSchema: { json: any }; + }; +} + +export interface BedrockToolConfiguration { + tools?: BedrockTool[]; + toolChoice?: + | { tool: { name: string } } + | { auto: {} } + | { any: {} } + | undefined; +} + +export const BEDROCK_STOP_REASONS = [ + 'stop', + 'stop_sequence', + 'end_turn', + 'length', + 'max_tokens', + 'content-filter', + 'content_filtered', + 'guardrail_intervened', + 'tool-calls', + 'tool_use', +] as const; + +export type BedrockStopReason = + | (typeof BEDROCK_STOP_REASONS)[number] + | (string & {}); + +export type BedrockImageFormat = 'jpeg' | 'png' | 'gif'; +export type BedrockDocumentFormat = 'pdf' | 'txt' | 'md'; + +export interface BedrockDocumentBlock { + document: { + format: BedrockDocumentFormat; + name: string; + source: { + bytes: string; + }; + }; +} + +export interface BedrockGuardrailConverseContentBlock { + guardContent: any; +} + +export interface BedrockImageBlock { + image: { + format: BedrockImageFormat; + source: { + bytes: string; + }; + }; +} + +export interface BedrockToolResultBlock { + toolResult: { + toolUseId: string; + content: Array<{ text: string }>; + }; +} + +export interface BedrockToolUseBlock { + toolUse: { + toolUseId: string; + name: string; + input: Record; + }; +} + +export interface BedrockTextBlock { + text: string; +} + +export type BedrockContentBlock = + | BedrockDocumentBlock + | BedrockGuardrailConverseContentBlock + | BedrockImageBlock + | BedrockTextBlock + | BedrockToolResultBlock + | BedrockToolUseBlock; diff --git a/packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts b/packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts index 1b9ff1aef492..6998311770bf 100644 --- a/packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts +++ b/packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts @@ -1,35 +1,17 @@ import { LanguageModelV1Prompt } from '@ai-sdk/provider'; -import { mockClient } from 'aws-sdk-client-mock'; -import { createAmazonBedrock } from './bedrock-provider'; import { - BedrockRuntimeClient, - ConverseCommand, - ConverseStreamCommand, - ConverseStreamOutput, - ConverseStreamTrace, - StopReason, -} from '@aws-sdk/client-bedrock-runtime'; -import { - convertArrayToAsyncIterable, + createTestServer, convertReadableStreamToArray, } from '@ai-sdk/provider-utils/test'; +import { BedrockChatLanguageModel } from './bedrock-chat-language-model'; +import { vi } from 'vitest'; +import { FetchFunction } from '@ai-sdk/provider-utils'; const TEST_PROMPT: LanguageModelV1Prompt = [ { role: 'system', content: 'System Prompt' }, { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; -const bedrockMock = mockClient(BedrockRuntimeClient); - -const provider = createAmazonBedrock({ - region: 'us-east-1', - accessKeyId: 'test-access-key', - secretAccessKey: 'test-secret-key', - sessionToken: 'test-token-key', -}); - -const model = provider('anthropic.claude-3-haiku-20240307-v1:0'); - const mockTrace = { guardrail: { inputAssessment: { @@ -51,346 +33,155 @@ const mockTrace = { type: 'PROFANITY' as const, }, ], - customWords: undefined, }, }, }, }, -} as ConverseStreamTrace; - -describe('doGenerate', () => { - beforeEach(() => { - bedrockMock.reset(); - }); - - it('should extract text response', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'Hello, World!' }] }, - }, - }); - - const { text } = await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); - - expect(text).toStrictEqual('Hello, World!'); - }); - - it('should extract usage', async () => { - bedrockMock.on(ConverseCommand).resolves({ - usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, - }); - - const { usage } = await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); +}; - expect(usage).toStrictEqual({ - promptTokens: 4, - completionTokens: 34, - }); - }); - - it('should extract finish reason', async () => { - bedrockMock.on(ConverseCommand).resolves({ - stopReason: 'stop_sequence', - }); - - const response = await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); - - expect(response.finishReason).toStrictEqual('stop'); - }); +function createFakeFetch(customHeaders: Record): FetchFunction { + return async (input, init = {}) => { + // Ensure headers is a plain object, Headers instance, or array. + if (init.headers instanceof Headers) { + for (const [key, value] of Object.entries(customHeaders)) { + init.headers.set(key, value); + } + } else if (Array.isArray(init.headers)) { + for (const [key, value] of Object.entries(customHeaders)) { + init.headers.push([key, value]); + } + } else { + init.headers = { ...(init.headers || {}), ...customHeaders }; + } + // Delegate to the global fetch (MSW will intercept it). + return await globalThis.fetch(input, init); + }; +} - it('should support unknown finish reason', async () => { - bedrockMock.on(ConverseCommand).resolves({ - stopReason: 'eos' as StopReason, - }); - - const response = await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); +const fakeFetchWithAuth = createFakeFetch({ 'x-amz-auth': 'test-auth' }); - expect(response.finishReason).toStrictEqual('unknown'); - }); - - it('should pass the model and the messages', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'Testing' }] }, - }, - }); +const modelId = 'anthropic.claude-3-haiku-20240307-v1:0'; +const baseUrl = 'https://bedrock-runtime.us-east-1.amazonaws.com'; - await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); +const streamUrl = `${baseUrl}/model/${encodeURIComponent( + modelId, +)}/converse-stream`; +const generateUrl = `${baseUrl}/model/${encodeURIComponent(modelId)}/converse`; +const server = createTestServer({ + [generateUrl]: {}, + [streamUrl]: { + response: { + type: 'stream-chunks', + chunks: [], + }, + }, +}); - expect( - bedrockMock.commandCalls(ConverseCommand, { - modelId: 'anthropic.claude-3-haiku-20240307-v1:0', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - }).length, - ).toBe(1); - }); +beforeEach(() => { + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [], + }; +}); - it('should pass settings', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'Testing' }] }, - }, - }); +const model = new BedrockChatLanguageModel( + modelId, + {}, + { + baseUrl: () => baseUrl, + headers: {}, + fetch: fakeFetchWithAuth, + generateId: () => 'test-id', + }, +); - await provider('amazon.titan-tg1-large', { - additionalModelRequestFields: { top_k: 10 }, - }).doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - maxTokens: 100, - temperature: 0.5, - topP: 0.5, - }); +let mockOptions: { success: boolean; errorValue?: any } = { success: true }; - expect( - bedrockMock.commandCalls(ConverseCommand, { - modelId: 'amazon.titan-tg1-large', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - additionalModelRequestFields: { top_k: 10 }, - system: [{ text: 'System Prompt' }], - inferenceConfig: { - maxTokens: 100, - temperature: 0.5, - topP: 0.5, - }, - }).length, - ).toBe(1); +describe('doStream', () => { + beforeEach(() => { + mockOptions = { success: true, errorValue: undefined }; }); - it('should pass tool specification in object-tool mode', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'ignored' }] }, - }, - }); - - await provider('amazon.titan-tg1-large').doGenerate({ - inputFormat: 'prompt', - mode: { - type: 'object-tool', - tool: { - name: 'test-tool', - type: 'function', - parameters: { - type: 'object', - properties: { - property1: { type: 'string' }, - property2: { type: 'number' }, - }, - required: ['property1', 'property2'], - additionalProperties: false, - }, - }, - }, - prompt: TEST_PROMPT, - }); - - expect( - bedrockMock.commandCalls(ConverseCommand, { - modelId: 'amazon.titan-tg1-large', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - system: [{ text: 'System Prompt' }], - toolConfig: { - tools: [ - { - toolSpec: { - name: 'test-tool', - description: undefined, - inputSchema: { - json: { - type: 'object', - properties: { - property1: { type: 'string' }, - property2: { type: 'number' }, - }, - required: ['property1', 'property2'], - additionalProperties: false, - }, - }, - }, + vi.mock('./bedrock-event-stream-response-handler', () => ({ + createBedrockEventStreamResponseHandler: (schema: any) => { + return async ({ response }: { response: Response }) => { + let chunks: { success: boolean; value: any }[] = []; + if (mockOptions.success) { + const text = await response.text(); + chunks = text + .split('\n') + .filter(Boolean) + .map(chunk => ({ + success: true, + value: JSON.parse(chunk), + })); + } + const headers: Record = {}; + response.headers.forEach((value, key) => { + headers[key] = value; + }); + return { + responseHeaders: headers, + value: new ReadableStream({ + start(controller) { + if (mockOptions.success) { + chunks.forEach(chunk => controller.enqueue(chunk)); + } else { + controller.enqueue({ + success: false, + error: mockOptions.errorValue, + }); + } + controller.close(); }, - ], - }, - }).length, - ).toBe(1); - }); + }), + }; + }; + }, + })); - it('should support guardrails', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'Testing' }] }, - }, - }); + function setupMockEventStreamHandler( + options: { success?: boolean; errorValue?: any } = { success: true }, + ) { + mockOptions = { ...mockOptions, ...options }; + } - // GuardrailConfiguration - const result = await provider('amazon.titan-tg1-large').doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - providerMetadata: { - bedrock: { - guardrailConfig: { - guardrailIdentifier: '-1', - guardrailVersion: '1', - trace: 'enabled', + it('should stream text deltas with metadata and usage', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { text: 'Hello' }, }, - }, - }, - }); - - expect( - bedrockMock.commandCalls(ConverseCommand, { - modelId: 'amazon.titan-tg1-large', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - system: [{ text: 'System Prompt' }], - guardrailConfig: { - guardrailIdentifier: '-1', - guardrailVersion: '1', - trace: 'enabled', - }, - }).length, - ).toBe(1); - }); - - it('should include trace information in providerMetadata', async () => { - bedrockMock.on(ConverseCommand).resolves({ - trace: mockTrace, - }); - - const response = await model.doGenerate({ - inputFormat: 'prompt', - mode: { type: 'regular' }, - prompt: TEST_PROMPT, - }); - - expect(response.providerMetadata?.bedrock.trace).toMatchObject(mockTrace); - }); - - it('should pass tools and tool choice correctly', async () => { - bedrockMock.on(ConverseCommand).resolves({ - output: { - message: { role: 'assistant', content: [{ text: 'Testing' }] }, - }, - }); - - await provider('amazon.titan-tg1-large').doGenerate({ - inputFormat: 'prompt', - mode: { - type: 'regular', - tools: [ - { - type: 'function', - name: 'test-tool-1', - description: 'A test tool', - parameters: { - type: 'object', - properties: { - param1: { type: 'string' }, - param2: { type: 'number' }, - }, - required: ['param1'], - additionalProperties: false, - }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 1, + delta: { text: ', ' }, }, - { - type: 'provider-defined', - name: 'unsupported-tool', - id: 'provider.unsupported-tool', - args: {}, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 2, + delta: { text: 'World!' }, }, - ], - toolChoice: { type: 'auto' }, - }, - prompt: TEST_PROMPT, - }); - - const calls = bedrockMock.commandCalls(ConverseCommand); - expect(calls.length).toBe(1); - expect(calls[0].args[0].input).toStrictEqual({ - additionalModelRequestFields: undefined, - guardrailConfig: undefined, - inferenceConfig: { - maxTokens: undefined, - stopSequences: undefined, - temperature: undefined, - topP: undefined, - }, - modelId: 'amazon.titan-tg1-large', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - system: [{ text: 'System Prompt' }], - toolConfig: { - tools: [ - { - toolSpec: { - name: 'test-tool-1', - description: 'A test tool', - inputSchema: { - json: { - type: 'object', - properties: { - param1: { type: 'string' }, - param2: { type: 'number' }, - }, - required: ['param1'], - additionalProperties: false, - }, - }, - }, + }) + '\n', + JSON.stringify({ + metadata: { + usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, + metrics: { latencyMs: 10 }, }, - ], - toolChoice: { auto: {} }, - }, - }); - }); -}); - -describe('doStream', () => { - beforeEach(() => { - bedrockMock.reset(); - }); - - it('should stream text deltas', async () => { - const streamData: ConverseStreamOutput[] = [ - { contentBlockDelta: { contentBlockIndex: 0, delta: { text: 'Hello' } } }, - { contentBlockDelta: { contentBlockIndex: 1, delta: { text: ', ' } } }, - { - contentBlockDelta: { contentBlockIndex: 2, delta: { text: 'World!' } }, - }, - { - metadata: { - usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, - metrics: { latencyMs: 10 }, - }, - }, - { - messageStop: { stopReason: 'stop_sequence' }, - }, - ]; - - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable(streamData), - }); + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'stop_sequence', + }, + }) + '\n', + ], + }; const { stream } = await model.doStream({ inputFormat: 'prompt', @@ -406,38 +197,45 @@ describe('doStream', () => { type: 'finish', finishReason: 'stop', usage: { promptTokens: 4, completionTokens: 34 }, - providerMetadata: undefined, }, ]); }); it('should stream tool deltas', async () => { - const streamData: ConverseStreamOutput[] = [ - { - contentBlockStart: { - contentBlockIndex: 0, - start: { toolUse: { toolUseId: 'tool-use-id', name: 'test-tool' } }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 0, - delta: { toolUse: { input: '{"value":' } }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 0, - delta: { toolUse: { input: '"Sparkle Day"}' } }, - }, - }, - { contentBlockStop: { contentBlockIndex: 0 } }, - { messageStop: { stopReason: 'tool_use' } }, - ]; - - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable(streamData), - }); + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + contentBlockStart: { + contentBlockIndex: 0, + start: { + toolUse: { toolUseId: 'tool-use-id', name: 'test-tool' }, + }, + }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { toolUse: { input: '{"value":' } }, + }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { toolUse: { input: '"Sparkle Day"}' } }, + }, + }) + '\n', + JSON.stringify({ + contentBlockStop: { contentBlockIndex: 0 }, + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'tool_use', + }, + }) + '\n', + ], + }; const { stream } = await model.doStream({ inputFormat: 'prompt', @@ -487,62 +285,69 @@ describe('doStream', () => { type: 'finish', finishReason: 'tool-calls', usage: { promptTokens: NaN, completionTokens: NaN }, - providerMetadata: undefined, }, ]); }); it('should stream parallel tool calls', async () => { - const streamData: ConverseStreamOutput[] = [ - { - contentBlockStart: { - contentBlockIndex: 0, - start: { - toolUse: { toolUseId: 'tool-use-id-1', name: 'test-tool-1' }, + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + contentBlockStart: { + contentBlockIndex: 0, + start: { + toolUse: { toolUseId: 'tool-use-id-1', name: 'test-tool-1' }, + }, }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 0, - delta: { toolUse: { input: '{"value1":' } }, - }, - }, - { - contentBlockStart: { - contentBlockIndex: 1, - start: { - toolUse: { toolUseId: 'tool-use-id-2', name: 'test-tool-2' }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { toolUse: { input: '{"value1":' } }, }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 1, - delta: { toolUse: { input: '{"value2":' } }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 1, - delta: { toolUse: { input: '"Sparkle Day"}' } }, - }, - }, - { - contentBlockDelta: { - contentBlockIndex: 0, - delta: { toolUse: { input: '"Sparkle Day"}' } }, - }, - }, - { contentBlockStop: { contentBlockIndex: 0 } }, - { contentBlockStop: { contentBlockIndex: 1 } }, - { messageStop: { stopReason: 'tool_use' } }, - ]; - - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable(streamData), - }); - + }) + '\n', + JSON.stringify({ + contentBlockStart: { + contentBlockIndex: 1, + start: { + toolUse: { toolUseId: 'tool-use-id-2', name: 'test-tool-2' }, + }, + }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 1, + delta: { toolUse: { input: '{"value2":' } }, + }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 1, + delta: { toolUse: { input: '"Sparkle Day"}' } }, + }, + }) + '\n', + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { toolUse: { input: '"Sparkle Day"}' } }, + }, + }) + '\n', + JSON.stringify({ + contentBlockStop: { contentBlockIndex: 0 }, + }) + '\n', + JSON.stringify({ + contentBlockStop: { contentBlockIndex: 1 }, + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'tool_use', + }, + }) + '\n', + ], + }; + const { stream } = await model.doStream({ inputFormat: 'prompt', mode: { @@ -623,24 +428,25 @@ describe('doStream', () => { type: 'finish', finishReason: 'tool-calls', usage: { promptTokens: NaN, completionTokens: NaN }, - providerMetadata: undefined, }, ]); }); it('should handle error stream parts', async () => { - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable([ - { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ internalServerException: { message: 'Internal Server Error', name: 'InternalServerException', $fault: 'server', $metadata: {}, }, - }, - ]), - }); + }) + '\n', + ], + }; const { stream } = await model.doStream({ inputFormat: 'prompt', @@ -648,7 +454,8 @@ describe('doStream', () => { prompt: TEST_PROMPT, }); - expect(await convertReadableStreamToArray(stream)).toStrictEqual([ + const result = await convertReadableStreamToArray(stream); + expect(result).toStrictEqual([ { type: 'error', error: { @@ -665,36 +472,185 @@ describe('doStream', () => { completionTokens: NaN, promptTokens: NaN, }, - providerMetadata: undefined, }, ]); }); - it('should pass the messages and the model', async () => { - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable([]), + it('should handle modelStreamErrorException error', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + modelStreamErrorException: { + message: 'Model Stream Error', + name: 'ModelStreamErrorException', + $fault: 'server', + $metadata: {}, + }, + }) + '\n', + ], + }; + + const { stream } = await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + const result = await convertReadableStreamToArray(stream); + expect(result).toStrictEqual([ + { + type: 'error', + error: { + message: 'Model Stream Error', + name: 'ModelStreamErrorException', + $fault: 'server', + $metadata: {}, + }, + }, + { + finishReason: 'error', + type: 'finish', + usage: { promptTokens: NaN, completionTokens: NaN }, + }, + ]); + }); + + it('should handle throttlingException error', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + throttlingException: { + message: 'Throttling Error', + name: 'ThrottlingException', + $fault: 'server', + $metadata: {}, + }, + }) + '\n', + ], + }; + + const { stream } = await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, }); + const result = await convertReadableStreamToArray(stream); + expect(result).toStrictEqual([ + { + type: 'error', + error: { + message: 'Throttling Error', + name: 'ThrottlingException', + $fault: 'server', + $metadata: {}, + }, + }, + { + finishReason: 'error', + type: 'finish', + usage: { promptTokens: NaN, completionTokens: NaN }, + }, + ]); + }); + + it('should handle validationException error', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + validationException: { + message: 'Validation Error', + name: 'ValidationException', + $fault: 'server', + $metadata: {}, + }, + }) + '\n', + ], + }; + + const { stream } = await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + const result = await convertReadableStreamToArray(stream); + expect(result).toStrictEqual([ + { + type: 'error', + error: { + message: 'Validation Error', + name: 'ValidationException', + $fault: 'server', + $metadata: {}, + }, + }, + { + finishReason: 'error', + type: 'finish', + usage: { promptTokens: NaN, completionTokens: NaN }, + }, + ]); + }); + + it('should handle failed chunk parsing', async () => { + setupMockEventStreamHandler({ + success: false, + errorValue: { message: 'Chunk Parsing Failed' }, + }); + + const { stream } = await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + const result = await convertReadableStreamToArray(stream); + expect(result).toStrictEqual([ + { + type: 'error', + error: { message: 'Chunk Parsing Failed' }, + }, + { + finishReason: 'error', + type: 'finish', + usage: { promptTokens: NaN, completionTokens: NaN }, + }, + ]); + }); + + it('should pass the messages and the model', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [], + }; + await model.doStream({ inputFormat: 'prompt', mode: { type: 'regular' }, prompt: TEST_PROMPT, }); - expect( - bedrockMock.commandCalls(ConverseStreamCommand, { - modelId: 'anthropic.claude-3-haiku-20240307-v1:0', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - }).length, - ).toBe(1); + expect(await server.calls[0].requestBody).toStrictEqual({ + messages: [{ role: 'user', content: [{ text: 'Hello' }] }], + system: [{ text: 'System Prompt' }], + }); }); it('should support guardrails', async () => { - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable([]), - }); + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [], + }; - await provider('amazon.titan-tg1-large').doStream({ + await model.doStream({ inputFormat: 'prompt', mode: { type: 'regular' }, prompt: TEST_PROMPT, @@ -710,37 +666,43 @@ describe('doStream', () => { }, }); - expect( - bedrockMock.commandCalls(ConverseStreamCommand, { - modelId: 'amazon.titan-tg1-large', - messages: [{ role: 'user', content: [{ text: 'Hello' }] }], - system: [{ text: 'System Prompt' }], - guardrailConfig: { - guardrailIdentifier: '-1', - guardrailVersion: '1', - trace: 'enabled', - streamProcessingMode: 'async', - }, - }).length, - ).toBe(1); + expect(await server.calls[0].requestBody).toMatchObject({ + messages: [{ role: 'user', content: [{ text: 'Hello' }] }], + system: [{ text: 'System Prompt' }], + guardrailConfig: { + guardrailIdentifier: '-1', + guardrailVersion: '1', + trace: 'enabled', + streamProcessingMode: 'async', + }, + }); }); it('should include trace information in providerMetadata', async () => { - const streamData: ConverseStreamOutput[] = [ - { contentBlockDelta: { contentBlockIndex: 0, delta: { text: 'Hello' } } }, - { - metadata: { - usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, - metrics: { latencyMs: 10 }, - trace: mockTrace, - }, - }, - { messageStop: { stopReason: 'stop_sequence' } }, - ]; - - bedrockMock.on(ConverseStreamCommand).resolves({ - stream: convertArrayToAsyncIterable(streamData), - }); + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { text: 'Hello' }, + }, + }) + '\n', + JSON.stringify({ + metadata: { + usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, + metrics: { latencyMs: 10 }, + trace: mockTrace, + }, + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'stop_sequence', + }, + }) + '\n', + ], + }; const { stream } = await model.doStream({ inputFormat: 'prompt', @@ -753,9 +715,589 @@ describe('doStream', () => { { type: 'finish', finishReason: 'stop', - usage: { completionTokens: 34, promptTokens: 4 }, - providerMetadata: { bedrock: { trace: mockTrace } }, + usage: { promptTokens: 4, completionTokens: 34 }, + providerMetadata: { + bedrock: { + trace: mockTrace, + }, + }, }, ]); }); + + it('should include response headers in rawResponse', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + headers: { + 'x-amzn-requestid': 'test-request-id', + 'x-amzn-trace-id': 'test-trace-id', + }, + chunks: [ + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { text: 'Hello' }, + }, + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'stop_sequence', + }, + }) + '\n', + ], + }; + + const response = await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(response.rawResponse?.headers).toEqual({ + 'cache-control': 'no-cache', + connection: 'keep-alive', + 'content-type': 'text/event-stream', + 'x-amzn-requestid': 'test-request-id', + 'x-amzn-trace-id': 'test-trace-id', + }); + }); + + it('should properly combine headers from all sources', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + headers: { + 'x-amzn-requestid': 'test-request-id', + 'x-amzn-trace-id': 'test-trace-id', + }, + chunks: [ + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { text: 'Hello' }, + }, + }) + '\n', + JSON.stringify({ + messageStop: { + stopReason: 'stop_sequence', + }, + }) + '\n', + ], + }; + + const optionsHeaders = { + 'options-header': 'options-value', + 'shared-header': 'options-shared', + }; + + const model = new BedrockChatLanguageModel( + modelId, + {}, + { + baseUrl: () => baseUrl, + headers: { + 'model-header': 'model-value', + 'shared-header': 'model-shared', + }, + fetch: createFakeFetch({ + 'options-header': 'options-value', + 'model-header': 'model-value', + 'shared-header': 'options-shared', + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + generateId: () => 'test-id', + }, + ); + + await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + headers: optionsHeaders, + }); + + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['options-header']).toBe('options-value'); + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); + expect(requestHeaders['shared-header']).toBe('options-shared'); + }); + + it('should work with partial headers', async () => { + setupMockEventStreamHandler(); + const model = new BedrockChatLanguageModel( + modelId, + {}, + { + baseUrl: () => baseUrl, + headers: { + 'model-header': 'model-value', + }, + fetch: createFakeFetch({ + 'model-header': 'model-value', + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + generateId: () => 'test-id', + }, + ); + + await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); + }); + + it('should include providerOptions in the request for streaming calls', async () => { + setupMockEventStreamHandler(); + server.urls[streamUrl].response = { + type: 'stream-chunks', + chunks: [ + JSON.stringify({ + contentBlockDelta: { + contentBlockIndex: 0, + delta: { text: 'Dummy' }, + }, + }) + '\n', + JSON.stringify({ + messageStop: { stopReason: 'stop_sequence' }, + }) + '\n', + ], + }; + + await model.doStream({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + providerMetadata: { + bedrock: { + foo: 'bar', + }, + }, + }); + + // Verify the outgoing request body includes "foo" at the top level. + const body = await server.calls[0].requestBody; + expect(body).toMatchObject({ foo: 'bar' }); + }); +}); + +describe('doGenerate', () => { + function prepareJsonResponse({ + content = 'Hello, World!', + toolCalls = [], + usage = { + inputTokens: 4, + outputTokens: 34, + totalTokens: 38, + }, + stopReason = 'stop_sequence', + trace, + }: { + content?: string; + toolCalls?: Array<{ + id?: string; + name: string; + args: Record; + }>; + usage?: { + inputTokens: number; + outputTokens: number; + totalTokens: number; + }; + stopReason?: string; + trace?: typeof mockTrace; + }) { + server.urls[generateUrl].response = { + type: 'json-value', + body: { + output: { + message: { + role: 'assistant', + content: [ + { type: 'text', text: content }, + ...toolCalls.map(tool => ({ + type: 'tool_use', + toolUseId: tool.id ?? 'tool-use-id', + name: tool.name, + input: tool.args, + })), + ], + }, + }, + usage, + stopReason, + ...(trace ? { trace } : {}), + }, + }; + } + + it('should extract text response', async () => { + prepareJsonResponse({ content: 'Hello, World!' }); + + const { text } = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(text).toStrictEqual('Hello, World!'); + }); + + it('should extract usage', async () => { + prepareJsonResponse({ + usage: { inputTokens: 4, outputTokens: 34, totalTokens: 38 }, + }); + + const { usage } = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(usage).toStrictEqual({ + promptTokens: 4, + completionTokens: 34, + }); + }); + + it('should extract finish reason', async () => { + prepareJsonResponse({ stopReason: 'stop_sequence' }); + + const { finishReason } = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(finishReason).toStrictEqual('stop'); + }); + + it('should support unknown finish reason', async () => { + prepareJsonResponse({ stopReason: 'eos' }); + + const { finishReason } = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(finishReason).toStrictEqual('unknown'); + }); + + it('should pass the model and the messages', async () => { + prepareJsonResponse({}); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(await server.calls[0].requestBody).toStrictEqual({ + messages: [{ role: 'user', content: [{ text: 'Hello' }] }], + system: [{ text: 'System Prompt' }], + }); + }); + + it('should pass settings', async () => { + prepareJsonResponse({}); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + maxTokens: 100, + temperature: 0.5, + topP: 0.5, + }); + + expect(await server.calls[0].requestBody).toMatchObject({ + inferenceConfig: { + maxTokens: 100, + temperature: 0.5, + topP: 0.5, + }, + }); + }); + + it('should pass tool specification in object-tool mode', async () => { + prepareJsonResponse({}); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { + type: 'object-tool', + tool: { + name: 'test-tool', + type: 'function', + parameters: { + type: 'object', + properties: { + property1: { type: 'string' }, + property2: { type: 'number' }, + }, + required: ['property1', 'property2'], + additionalProperties: false, + }, + }, + }, + prompt: TEST_PROMPT, + }); + + expect(await server.calls[0].requestBody).toMatchObject({ + toolConfig: { + tools: [ + { + toolSpec: { + name: 'test-tool', + inputSchema: { + json: { + type: 'object', + properties: { + property1: { type: 'string' }, + property2: { type: 'number' }, + }, + required: ['property1', 'property2'], + additionalProperties: false, + }, + }, + }, + }, + ], + }, + }); + }); + + it('should support guardrails', async () => { + prepareJsonResponse({}); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + providerMetadata: { + bedrock: { + guardrailConfig: { + guardrailIdentifier: '-1', + guardrailVersion: '1', + trace: 'enabled', + }, + }, + }, + }); + + expect(await server.calls[0].requestBody).toMatchObject({ + guardrailConfig: { + guardrailIdentifier: '-1', + guardrailVersion: '1', + trace: 'enabled', + }, + }); + }); + + it('should include trace information in providerMetadata', async () => { + prepareJsonResponse({ trace: mockTrace }); + + const response = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(response.providerMetadata?.bedrock.trace).toMatchObject(mockTrace); + }); + + it('should include response headers in rawResponse', async () => { + server.urls[generateUrl].response = { + type: 'json-value', + headers: { + 'x-amzn-requestid': 'test-request-id', + 'x-amzn-trace-id': 'test-trace-id', + }, + body: { + output: { + message: { + role: 'assistant', + content: [{ text: 'Testing' }], + }, + }, + usage: { + inputTokens: 4, + outputTokens: 34, + totalTokens: 38, + }, + stopReason: 'stop_sequence', + }, + }; + + const response = await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + expect(response.rawResponse?.headers).toEqual({ + 'x-amzn-requestid': 'test-request-id', + 'x-amzn-trace-id': 'test-trace-id', + 'content-type': 'application/json', + 'content-length': '164', + }); + }); + + it('should pass tools and tool choice correctly', async () => { + prepareJsonResponse({}); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { + type: 'regular', + tools: [ + { + type: 'function', + name: 'test-tool-1', + description: 'A test tool', + parameters: { + type: 'object', + properties: { + param1: { type: 'string' }, + param2: { type: 'number' }, + }, + required: ['param1'], + additionalProperties: false, + }, + }, + ], + toolChoice: { type: 'auto' }, + }, + prompt: TEST_PROMPT, + }); + + expect(await server.calls[0].requestBody).toMatchObject({ + toolConfig: { + tools: [ + { + toolSpec: { + name: 'test-tool-1', + description: 'A test tool', + inputSchema: { + json: { + type: 'object', + properties: { + param1: { type: 'string' }, + param2: { type: 'number' }, + }, + required: ['param1'], + additionalProperties: false, + }, + }, + }, + }, + ], + }, + }); + }); + + it('should properly combine headers from all sources', async () => { + prepareJsonResponse({}); + + const optionsHeaders = { + 'options-header': 'options-value', + 'shared-header': 'options-shared', + }; + + const model = new BedrockChatLanguageModel( + modelId, + {}, + { + baseUrl: () => baseUrl, + headers: { + 'model-header': 'model-value', + 'shared-header': 'model-shared', + }, + fetch: createFakeFetch({ + 'options-header': 'options-value', + 'model-header': 'model-value', + 'shared-header': 'options-shared', + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + generateId: () => 'test-id', + }, + ); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + headers: optionsHeaders, + }); + + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['options-header']).toBe('options-value'); + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); + expect(requestHeaders['shared-header']).toBe('options-shared'); + }); + + it('should work with partial headers', async () => { + prepareJsonResponse({}); + + const model = new BedrockChatLanguageModel( + modelId, + {}, + { + baseUrl: () => baseUrl, + headers: { + 'model-header': 'model-value', + }, + fetch: createFakeFetch({ + 'model-header': 'model-value', + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + generateId: () => 'test-id', + }, + ); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + }); + + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); + }); + + it('should include providerOptions in the request for generate calls', async () => { + prepareJsonResponse({ content: 'Test generation' }); + + await model.doGenerate({ + inputFormat: 'prompt', + mode: { type: 'regular' }, + prompt: TEST_PROMPT, + providerMetadata: { + bedrock: { + foo: 'bar', + }, + }, + }); + + // Verify that the outgoing request body includes "foo" at its top level. + const body = await server.calls[0].requestBody; + expect(body).toMatchObject({ foo: 'bar' }); + }); }); diff --git a/packages/amazon-bedrock/src/bedrock-chat-language-model.ts b/packages/amazon-bedrock/src/bedrock-chat-language-model.ts index fbcc4aafa8d9..e621ad102d56 100644 --- a/packages/amazon-bedrock/src/bedrock-chat-language-model.ts +++ b/packages/amazon-bedrock/src/bedrock-chat-language-model.ts @@ -7,27 +7,36 @@ import { LanguageModelV1StreamPart, UnsupportedFunctionalityError, } from '@ai-sdk/provider'; -import { ParseResult } from '@ai-sdk/provider-utils'; import { - BedrockRuntimeClient, - ConverseCommand, - ConverseCommandInput, - ConverseStreamCommand, - ConverseStreamOutput, - GuardrailConfiguration, - GuardrailStreamConfiguration, - ToolInputSchema, -} from '@aws-sdk/client-bedrock-runtime'; + FetchFunction, + ParseResult, + Resolvable, + combineHeaders, + createJsonErrorResponseHandler, + createJsonResponseHandler, + postJsonToApi, + resolve, +} from '@ai-sdk/provider-utils'; +import { + BedrockConverseInput, + BEDROCK_STOP_REASONS, + BedrockToolInputSchema, +} from './bedrock-api-types'; import { BedrockChatModelId, BedrockChatSettings, } from './bedrock-chat-settings'; +import { BedrockErrorSchema } from './bedrock-error'; +import { createBedrockEventStreamResponseHandler } from './bedrock-event-stream-response-handler'; import { prepareTools } from './bedrock-prepare-tools'; import { convertToBedrockChatMessages } from './convert-to-bedrock-chat-messages'; import { mapBedrockFinishReason } from './map-bedrock-finish-reason'; +import { z } from 'zod'; type BedrockChatConfig = { - client: BedrockRuntimeClient; + baseUrl: () => string; + headers: Resolvable>; + fetch?: FetchFunction; generateId: () => string; }; @@ -37,20 +46,11 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { readonly defaultObjectGenerationMode = 'tool'; readonly supportsImageUrls = false; - readonly modelId: BedrockChatModelId; - readonly settings: BedrockChatSettings; - - private readonly config: BedrockChatConfig; - constructor( - modelId: BedrockChatModelId, - settings: BedrockChatSettings, - config: BedrockChatConfig, - ) { - this.modelId = modelId; - this.settings = settings; - this.config = config; - } + readonly modelId: BedrockChatModelId, + private readonly settings: BedrockChatSettings, + private readonly config: BedrockChatConfig, + ) {} private getArgs({ mode, @@ -67,7 +67,7 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { providerMetadata, headers, }: Parameters[0]): { - command: ConverseCommandInput; + command: BedrockConverseInput; warnings: LanguageModelV1CallWarning[]; } { const type = mode.type; @@ -95,13 +95,6 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { }); } - if (headers != null) { - warnings.push({ - type: 'unsupported-setting', - setting: 'headers', - }); - } - if (topK != null) { warnings.push({ type: 'unsupported-setting', @@ -109,6 +102,7 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { }); } + // TODO: validate this is still the case. if (responseFormat != null && responseFormat.type !== 'text') { warnings.push({ type: 'unsupported-setting', @@ -119,21 +113,21 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { const { system, messages } = convertToBedrockChatMessages(prompt); - const baseArgs: ConverseCommandInput = { - modelId: this.modelId, + const inferenceConfig = { + ...(maxTokens != null && { maxTokens }), + ...(temperature != null && { temperature }), + ...(topP != null && { topP }), + ...(stopSequences != null && { stopSequences }), + }; + + const baseArgs: BedrockConverseInput = { system: system ? [{ text: system }] : undefined, additionalModelRequestFields: this.settings.additionalModelRequestFields, - inferenceConfig: { - maxTokens, - temperature, - topP, - stopSequences, - }, + ...(Object.keys(inferenceConfig).length > 0 && { + inferenceConfig, + }), messages, - guardrailConfig: providerMetadata?.bedrock?.guardrailConfig as - | GuardrailConfiguration - | GuardrailStreamConfiguration - | undefined, + ...providerMetadata?.bedrock, }; switch (type) { @@ -166,7 +160,7 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { description: mode.tool.description, inputSchema: { json: mode.tool.parameters, - } as ToolInputSchema, + } as BedrockToolInputSchema, }, }, ], @@ -187,13 +181,28 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { async doGenerate( options: Parameters[0], ): Promise>> { - const { command, warnings } = this.getArgs(options); - - const response = await this.config.client.send( - new ConverseCommand(command), - ); + const { command: args, warnings } = this.getArgs(options); + + const url = this.getUrl(this.modelId); + const { value: response, responseHeaders } = await postJsonToApi({ + url, + headers: combineHeaders( + await resolve(this.config.headers), + options.headers, + ), + body: args, + failedResponseHandler: createJsonErrorResponseHandler({ + errorSchema: BedrockErrorSchema, + errorToMessage: error => `${error.message ?? 'Unknown error'}`, + }), + successfulResponseHandler: createJsonResponseHandler( + BedrockResponseSchema, + ), + abortSignal: options.abortSignal, + fetch: this.config.fetch, + }); - const { messages: rawPrompt, ...rawSettings } = command; + const { messages: rawPrompt, ...rawSettings } = args; const providerMetadata = response.trace ? { bedrock: { trace: response.trace as JSONObject } } @@ -218,43 +227,45 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { completionTokens: response.usage?.outputTokens ?? Number.NaN, }, rawCall: { rawPrompt, rawSettings }, + rawResponse: { headers: responseHeaders }, warnings, - providerMetadata, + ...(providerMetadata && { providerMetadata }), }; } async doStream( options: Parameters[0], ): Promise>> { - const { command, warnings } = this.getArgs(options); - - const response = await this.config.client.send( - new ConverseStreamCommand(command), - ); + const { command: args, warnings } = this.getArgs(options); + const url = this.getStreamUrl(this.modelId); + + const { value: response, responseHeaders } = await postJsonToApi({ + url, + headers: combineHeaders( + await resolve(this.config.headers), + options.headers, + ), + body: args, + failedResponseHandler: createJsonErrorResponseHandler({ + errorSchema: BedrockErrorSchema, + errorToMessage: error => `${error.type}: ${error.message}`, + }), + successfulResponseHandler: + createBedrockEventStreamResponseHandler(BedrockStreamSchema), + abortSignal: options.abortSignal, + fetch: this.config.fetch, + }); - const { messages: rawPrompt, ...rawSettings } = command; + const { messages: rawPrompt, ...rawSettings } = args; let finishReason: LanguageModelV1FinishReason = 'unknown'; - let usage: { promptTokens: number; completionTokens: number } = { + let usage = { promptTokens: Number.NaN, completionTokens: Number.NaN, }; let providerMetadata: LanguageModelV1ProviderMetadata | undefined = undefined; - if (!response.stream) { - throw new Error('No stream found'); - } - - const stream = new ReadableStream({ - async start(controller) { - for await (const chunk of response.stream!) { - controller.enqueue({ success: true, value: chunk }); - } - controller.close(); - }, - }); - const toolCallContentBlocks: Record< number, { @@ -265,15 +276,15 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { > = {}; return { - stream: stream.pipeThrough( + stream: response.pipeThrough( new TransformStream< - ParseResult, + ParseResult>, LanguageModelV1StreamPart >({ transform(chunk, controller) { - function enqueueError(error: Error) { + function enqueueError(bedrockError: Record) { finishReason = 'error'; - controller.enqueue({ type: 'error', error }); + controller.enqueue({ type: 'error', error: bedrockError }); } // handle failed chunk parsing / validation: @@ -377,19 +388,111 @@ export class BedrockChatLanguageModel implements LanguageModelV1 { } } }, - flush(controller) { controller.enqueue({ type: 'finish', finishReason, usage, - providerMetadata, + ...(providerMetadata && { providerMetadata }), }); }, }), ), rawCall: { rawPrompt, rawSettings }, + rawResponse: { headers: responseHeaders }, warnings, }; } + + private getUrl(modelId: string) { + const encodedModelId = encodeURIComponent(modelId); + return `${this.config.baseUrl()}/model/${encodedModelId}/converse`; + } + + private getStreamUrl(modelId: string): string { + const encodedModelId = encodeURIComponent(modelId); + return `${this.config.baseUrl()}/model/${encodedModelId}/converse-stream`; + } } + +const BedrockStopReasonSchema = z.union([ + z.enum(BEDROCK_STOP_REASONS), + z.string(), +]); + +// limited version of the schema, focussed on what is needed for the implementation +// this approach limits breakages when the API changes and increases efficiency +const BedrockResponseSchema = z.object({ + metrics: z + .object({ + latencyMs: z.number(), + }) + .nullish(), + output: z.object({ + message: z.object({ + content: z.array( + z.object({ + text: z.string().optional(), + toolUse: z + .object({ + toolUseId: z.string(), + name: z.string(), + input: z.any(), + }) + .optional(), + }), + ), + role: z.string(), + }), + }), + stopReason: BedrockStopReasonSchema, + trace: z.any().nullish(), + usage: z.object({ + inputTokens: z.number(), + outputTokens: z.number(), + totalTokens: z.number(), + }), +}); + +// limited version of the schema, focussed on what is needed for the implementation +// this approach limits breakages when the API changes and increases efficiency +const BedrockStreamSchema = z.object({ + contentBlockDelta: z + .object({ + contentBlockIndex: z.number(), + delta: z.record(z.any()).nullish(), + }) + .nullish(), + contentBlockStart: z + .object({ + contentBlockIndex: z.number(), + start: z.record(z.any()).nullish(), + }) + .nullish(), + contentBlockStop: z + .object({ + contentBlockIndex: z.number(), + }) + .nullish(), + internalServerException: z.record(z.any()).nullish(), + messageStop: z + .object({ + additionalModelResponseFields: z.any().nullish(), + stopReason: BedrockStopReasonSchema, + }) + .nullish(), + metadata: z + .object({ + trace: z.any(), + usage: z + .object({ + inputTokens: z.number(), + outputTokens: z.number(), + }) + .nullish(), + }) + .nullish(), + modelStreamErrorException: z.record(z.any()).nullish(), + throttlingException: z.record(z.any()).nullish(), + validationException: z.record(z.any()).nullish(), +}); diff --git a/packages/amazon-bedrock/src/bedrock-chat-prompt.ts b/packages/amazon-bedrock/src/bedrock-chat-prompt.ts index e30297732667..c792f67517e1 100644 --- a/packages/amazon-bedrock/src/bedrock-chat-prompt.ts +++ b/packages/amazon-bedrock/src/bedrock-chat-prompt.ts @@ -1,4 +1,4 @@ -import { ContentBlock } from '@aws-sdk/client-bedrock-runtime'; +import { BedrockContentBlock } from './bedrock-api-types'; export type BedrockMessagesPrompt = { system?: string; @@ -11,10 +11,10 @@ export type BedrockMessage = BedrockUserMessage | BedrockAssistantMessage; export interface BedrockUserMessage { role: 'user'; - content: Array; + content: Array; } export interface BedrockAssistantMessage { role: 'assistant'; - content: Array; + content: Array; } diff --git a/packages/amazon-bedrock/src/bedrock-embedding-model.test.ts b/packages/amazon-bedrock/src/bedrock-embedding-model.test.ts index 061a7cca1b62..6af4d6223bad 100644 --- a/packages/amazon-bedrock/src/bedrock-embedding-model.test.ts +++ b/packages/amazon-bedrock/src/bedrock-embedding-model.test.ts @@ -1,11 +1,7 @@ -import { mockClient } from 'aws-sdk-client-mock'; +import { createTestServer } from '@ai-sdk/provider-utils/test'; import { createAmazonBedrock } from './bedrock-provider'; -import { - BedrockRuntimeClient, - InvokeModelCommand, -} from '@aws-sdk/client-bedrock-runtime'; - -const bedrockMock = mockClient(BedrockRuntimeClient); +import { BedrockEmbeddingModel } from './bedrock-embedding-model'; +import { FetchFunction } from '@ai-sdk/provider-utils'; const mockEmbeddings = [ [ @@ -18,122 +14,175 @@ const mockEmbeddings = [ ], ]; +// TODO: share with bedrock-chat-language-model.test.ts +function createFakeFetch(customHeaders: Record): FetchFunction { + return async (input, init = {}) => { + // Ensure headers is a plain object, Headers instance, or array. + if (init.headers instanceof Headers) { + for (const [key, value] of Object.entries(customHeaders)) { + init.headers.set(key, value); + } + } else if (Array.isArray(init.headers)) { + for (const [key, value] of Object.entries(customHeaders)) { + init.headers.push([key, value]); + } + } else { + init.headers = { ...(init.headers || {}), ...customHeaders }; + } + // Delegate to the global fetch (MSW will intercept it). + return await globalThis.fetch(input, init); + }; +} + +const fakeFetchWithAuth = createFakeFetch({ 'x-amz-auth': 'test-auth' }); + const testValues = ['sunny day at the beach', 'rainy day in the city']; -const provider = createAmazonBedrock({ - region: 'us-east-1', - accessKeyId: 'test-access-key', - secretAccessKey: 'test-secret-key', - sessionToken: 'test-token-key', -}); +const embedUrl = `https://bedrock-runtime.us-east-1.amazonaws.com/model/${encodeURIComponent( + 'amazon.titan-embed-text-v2:0', +)}/invoke`; describe('doEmbed', () => { - beforeEach(() => { - bedrockMock.reset(); + const mockConfigHeaders = { + 'config-header': 'config-value', + 'shared-header': 'config-shared', + }; + + const server = createTestServer({ + [embedUrl]: { + response: { + type: 'binary', + headers: { + 'content-type': 'application/json', + }, + body: Buffer.from( + JSON.stringify({ + embedding: mockEmbeddings[0], + inputTextTokenCount: 8, + }), + ), + }, + }, }); - it('should handle single input value and return embeddings', async () => { - const mockResponse = { - embedding: mockEmbeddings[0], - inputTextTokenCount: 8, + const model = new BedrockEmbeddingModel( + 'amazon.titan-embed-text-v2:0', + {}, + { + baseUrl: () => 'https://bedrock-runtime.us-east-1.amazonaws.com', + headers: mockConfigHeaders, + fetch: fakeFetchWithAuth, + }, + ); + + let callCount = 0; + + beforeEach(() => { + callCount = 0; + server.urls[embedUrl].response = { + type: 'binary', + headers: { + 'content-type': 'application/json', + }, + body: Buffer.from( + JSON.stringify({ + embedding: mockEmbeddings[0], + inputTextTokenCount: 8, + }), + ), }; + }); - bedrockMock.on(InvokeModelCommand).resolves({ - //@ts-ignore - body: new TextEncoder().encode(JSON.stringify(mockResponse)), + it('should handle single input value and return embeddings', async () => { + const { embeddings } = await model.doEmbed({ + values: [testValues[0]], }); - const { embeddings } = await provider - .embedding('amazon.titan-embed-text-v2:0') - .doEmbed({ - values: [testValues[0]], - }); - expect(embeddings.length).toBe(1); - expect(embeddings[0]).toStrictEqual(mockResponse.embedding); + expect(embeddings[0]).toStrictEqual(mockEmbeddings[0]); + + const body = await server.calls[0].requestBody; + expect(body).toEqual({ + inputText: testValues[0], + dimensions: undefined, + normalize: undefined, + }); }); it('should handle single input value and extract usage', async () => { - const mockResponse = { - embedding: [], - inputTextTokenCount: 8, - }; - - bedrockMock.on(InvokeModelCommand).resolves({ - //@ts-ignore - body: new TextEncoder().encode(JSON.stringify(mockResponse)), + const { usage } = await model.doEmbed({ + values: [testValues[0]], }); - const { usage } = await provider - .embedding('amazon.titan-embed-text-v2:0') - .doEmbed({ - values: [testValues[0]], - }); - expect(usage?.tokens).toStrictEqual(8); }); - it('should handle multiple input values and return embeddings', async () => { - bedrockMock - .on(InvokeModelCommand) - .resolvesOnce({ - //@ts-ignore - body: new TextEncoder().encode( - JSON.stringify({ - embedding: mockEmbeddings[0], - inputTextTokenCount: 8, - }), - ), - }) - .resolvesOnce({ - //@ts-ignore - body: new TextEncoder().encode( - JSON.stringify({ - embedding: mockEmbeddings[1], - inputTextTokenCount: 8, - }), - ), - }); - - const { embeddings } = await provider - .embedding('amazon.titan-embed-text-v2:0') - .doEmbed({ - values: testValues, - }); + it('should handle multiple input values and extract usage', async () => { + const { usage } = await model.doEmbed({ + values: testValues, + }); - expect(embeddings.length).toBe(2); - expect(embeddings[0]).toStrictEqual(mockEmbeddings[0]); - expect(embeddings[1]).toStrictEqual(mockEmbeddings[1]); + expect(usage?.tokens).toStrictEqual(16); }); - it('should handle multiple input values and extract usage', async () => { - bedrockMock - .on(InvokeModelCommand) - .resolvesOnce({ - //@ts-ignore - body: new TextEncoder().encode( - JSON.stringify({ - embedding: [], - inputTextTokenCount: 8, - }), - ), - }) - .resolvesOnce({ - //@ts-ignore - body: new TextEncoder().encode( - JSON.stringify({ - embedding: [], - inputTextTokenCount: 8, - }), - ), - }); + it('should properly combine headers from all sources', async () => { + const optionsHeaders = { + 'options-header': 'options-value', + 'shared-header': 'options-shared', + }; - const { usage } = await provider - .embedding('amazon.titan-embed-text-v2:0') - .doEmbed({ - values: testValues, - }); + const modelWithHeaders = new BedrockEmbeddingModel( + 'amazon.titan-embed-text-v2:0', + {}, + { + baseUrl: () => 'https://bedrock-runtime.us-east-1.amazonaws.com', + headers: { + 'model-header': 'model-value', + 'shared-header': 'model-shared', + }, + fetch: createFakeFetch({ + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + }, + ); + + await modelWithHeaders.doEmbed({ + values: [testValues[0]], + headers: optionsHeaders, + }); - expect(usage?.tokens).toStrictEqual(16); + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['options-header']).toBe('options-value'); + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); + expect(requestHeaders['shared-header']).toBe('options-shared'); + }); + + it('should work with partial headers', async () => { + const modelWithPartialHeaders = new BedrockEmbeddingModel( + 'amazon.titan-embed-text-v2:0', + {}, + { + baseUrl: () => 'https://bedrock-runtime.us-east-1.amazonaws.com', + headers: { + 'model-header': 'model-value', + }, + fetch: createFakeFetch({ + 'signed-header': 'signed-value', + authorization: 'AWS4-HMAC-SHA256...', + }), + }, + ); + + await modelWithPartialHeaders.doEmbed({ + values: [testValues[0]], + }); + + const requestHeaders = server.calls[0].requestHeaders; + expect(requestHeaders['model-header']).toBe('model-value'); + expect(requestHeaders['signed-header']).toBe('signed-value'); + expect(requestHeaders['authorization']).toBe('AWS4-HMAC-SHA256...'); }); }); diff --git a/packages/amazon-bedrock/src/bedrock-embedding-model.ts b/packages/amazon-bedrock/src/bedrock-embedding-model.ts index 68cbd7b3a986..d5d2d0cee8d2 100644 --- a/packages/amazon-bedrock/src/bedrock-embedding-model.ts +++ b/packages/amazon-bedrock/src/bedrock-embedding-model.ts @@ -1,73 +1,97 @@ -import { EmbeddingModelV1 } from '@ai-sdk/provider'; +import { EmbeddingModelV1, EmbeddingModelV1Embedding } from '@ai-sdk/provider'; +import { + FetchFunction, + Resolvable, + combineHeaders, + createJsonErrorResponseHandler, + postJsonToApi, + resolve, +} from '@ai-sdk/provider-utils'; import { BedrockEmbeddingModelId, BedrockEmbeddingSettings, } from './bedrock-embedding-settings'; -import { - BedrockRuntimeClient, - InvokeModelCommand, -} from '@aws-sdk/client-bedrock-runtime'; +import { BedrockErrorSchema } from './bedrock-error'; type BedrockEmbeddingConfig = { - client: BedrockRuntimeClient; + baseUrl: () => string; + headers: Resolvable>; + fetch?: FetchFunction; }; type DoEmbedResponse = Awaited['doEmbed']>>; export class BedrockEmbeddingModel implements EmbeddingModelV1 { readonly specificationVersion = 'v1'; - readonly modelId: BedrockEmbeddingModelId; readonly provider = 'amazon-bedrock'; readonly maxEmbeddingsPerCall = undefined; readonly supportsParallelCalls = true; - private readonly config: BedrockEmbeddingConfig; - private readonly settings: BedrockEmbeddingSettings; constructor( - modelId: BedrockEmbeddingModelId, - settings: BedrockEmbeddingSettings, - config: BedrockEmbeddingConfig, - ) { - this.modelId = modelId; - this.config = config; - this.settings = settings; + readonly modelId: BedrockEmbeddingModelId, + private readonly settings: BedrockEmbeddingSettings, + private readonly config: BedrockEmbeddingConfig, + ) {} + + private getUrl(modelId: string): string { + const encodedModelId = encodeURIComponent(modelId); + return `${this.config.baseUrl()}/model/${encodedModelId}/invoke`; } async doEmbed({ values, + headers, + abortSignal, }: Parameters< EmbeddingModelV1['doEmbed'] >[0]): Promise { - const fn = async (inputText: string) => { - const payload = { + const embedSingleText = async (inputText: string) => { + // https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html + const args = { inputText, dimensions: this.settings.dimensions, normalize: this.settings.normalize, }; - - const command = new InvokeModelCommand({ - contentType: 'application/json', - body: JSON.stringify(payload), - modelId: this.modelId, + const url = this.getUrl(this.modelId); + const { value: response } = await postJsonToApi({ + url, + headers: await resolve( + combineHeaders(await resolve(this.config.headers), headers), + ), + body: args, + failedResponseHandler: createJsonErrorResponseHandler({ + errorSchema: BedrockErrorSchema, + errorToMessage: error => `${error.type}: ${error.message}`, + }), + successfulResponseHandler: async response => { + const binaryData = await response.response.arrayBuffer(); + const jsonString = new TextDecoder().decode( + new Uint8Array(binaryData), + ); + const parsed = JSON.parse(jsonString); + return { value: parsed }; + }, + fetch: this.config.fetch, + abortSignal, }); - const rawResponse = await this.config.client.send(command); - const parsed = JSON.parse(new TextDecoder().decode(rawResponse.body)); - - return parsed; + return { + embedding: response.embedding, + inputTextTokenCount: response.inputTextTokenCount, + }; }; - const responses = await Promise.all(values.map(fn)); - - const response = responses.reduce( - (acc, r) => { - acc.embeddings.push(r.embedding); - acc.usage.tokens += r.inputTextTokenCount; - return acc; + const responses = await Promise.all(values.map(embedSingleText)); + return responses.reduce<{ + embeddings: EmbeddingModelV1Embedding[]; + usage: { tokens: number }; + }>( + (accumulated, response) => { + accumulated.embeddings.push(response.embedding); + accumulated.usage.tokens += response.inputTextTokenCount; + return accumulated; }, { embeddings: [], usage: { tokens: 0 } }, ); - - return response; } } diff --git a/packages/amazon-bedrock/src/bedrock-error.ts b/packages/amazon-bedrock/src/bedrock-error.ts new file mode 100644 index 000000000000..e4c419b14b14 --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-error.ts @@ -0,0 +1,6 @@ +import { z } from 'zod'; + +export const BedrockErrorSchema = z.object({ + message: z.string(), + type: z.string().nullish(), +}); diff --git a/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.test.ts b/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.test.ts new file mode 100644 index 000000000000..a1dbd24ee7c1 --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.test.ts @@ -0,0 +1,233 @@ +import { EmptyResponseBodyError } from '@ai-sdk/provider'; +import { createBedrockEventStreamResponseHandler } from './bedrock-event-stream-response-handler'; +import { EventStreamCodec } from '@smithy/eventstream-codec'; +import { z } from 'zod'; +import { describe, it, expect, vi, MockInstance } from 'vitest'; + +// Helper that constructs a properly framed message. +// The first 4 bytes will contain the frame total length (big-endian). +const createFrame = (payload: Uint8Array): Uint8Array => { + const totalLength = 4 + payload.length; + const frame = new Uint8Array(totalLength); + new DataView(frame.buffer).setUint32(0, totalLength, false); + frame.set(payload, 4); + return frame; +}; + +// Mock EventStreamCodec +vi.mock('@smithy/eventstream-codec', () => ({ + EventStreamCodec: vi.fn(), +})); + +describe('createEventSourceResponseHandler', () => { + // Define a sample schema for testing + const testSchema = z.object({ + chunk: z.object({ + content: z.string(), + }), + }); + + it('throws EmptyResponseBodyError when response body is null', async () => { + const response = new Response(null); + const handler = createBedrockEventStreamResponseHandler(testSchema); + + await expect( + handler({ + response, + url: 'test-url', + requestBodyValues: {}, + }), + ).rejects.toThrow(EmptyResponseBodyError); + }); + + it('successfully processes valid event stream data', async () => { + // Prepare the message we wish to simulate. + // Our decoded message will contain headers and a body that is valid JSON. + const message = { + headers: { + ':message-type': { value: 'event' }, + ':event-type': { value: 'chunk' }, + }, + body: new TextEncoder().encode( + JSON.stringify({ content: 'test message' }), + ), + }; + + // Create a frame that properly encapsulates the message. + const dummyPayload = new Uint8Array([1, 2, 3, 4]); // arbitrary payload that makes the length check pass + const frame = createFrame(dummyPayload); + + const mockDecode = vi.fn().mockReturnValue(message); + (EventStreamCodec as unknown as MockInstance).mockImplementation(() => ({ + decode: mockDecode, + })); + + // Create a stream that enqueues the complete frame. + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(frame); + controller.close(); + }, + }); + + const response = new Response(stream); + const handler = createBedrockEventStreamResponseHandler(testSchema); + const result = await handler({ + response, + url: 'test-url', + requestBodyValues: {}, + }); + + const reader = result.value.getReader(); + const { done, value } = await reader.read(); + + expect(done).toBe(false); + expect(value).toEqual({ + success: true, + value: { chunk: { content: 'test message' } }, + rawValue: { chunk: { content: 'test message' } }, + }); + }); + + it('handles invalid JSON data', async () => { + // Our mock decode returns a body that is not valid JSON. + const message = { + headers: { + ':message-type': { value: 'event' }, + ':event-type': { value: 'chunk' }, + }, + body: new TextEncoder().encode('invalid json'), + }; + + const dummyPayload = new Uint8Array([5, 6, 7, 8]); + const frame = createFrame(dummyPayload); + + const mockDecode = vi.fn().mockReturnValue(message); + (EventStreamCodec as unknown as MockInstance).mockImplementation(() => ({ + decode: mockDecode, + })); + + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(frame); + controller.close(); + }, + }); + + const response = new Response(stream); + const handler = createBedrockEventStreamResponseHandler(testSchema); + const result = await handler({ + response, + url: 'test-url', + requestBodyValues: {}, + }); + + const reader = result.value.getReader(); + const { done, value } = await reader.read(); + + expect(done).toBe(false); + // When JSON is invalid, safeParseJSON returns a result with success: false. + expect(value?.success).toBe(false); + expect((value as { success: false; error: Error }).error).toBeDefined(); + }); + + it('handles schema validation failures', async () => { + // The decoded message returns valid JSON but that does not meet our schema. + const message = { + headers: { + ':message-type': { value: 'event' }, + ':event-type': { value: 'chunk' }, + }, + body: new TextEncoder().encode(JSON.stringify({ invalid: 'data' })), + }; + + const dummyPayload = new Uint8Array([9, 10, 11, 12]); + const frame = createFrame(dummyPayload); + + const mockDecode = vi.fn().mockReturnValue(message); + (EventStreamCodec as unknown as MockInstance).mockImplementation(() => ({ + decode: mockDecode, + })); + + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(frame); + controller.close(); + }, + }); + + const response = new Response(stream); + const handler = createBedrockEventStreamResponseHandler(testSchema); + const result = await handler({ + response, + url: 'test-url', + requestBodyValues: {}, + }); + + const reader = result.value.getReader(); + const { done, value } = await reader.read(); + + expect(done).toBe(false); + // The schema does not match so safeParseJSON with the schema should yield success: false. + expect(value?.success).toBe(false); + expect((value as { success: false; error: Error }).error).toBeDefined(); + }); + + it('handles partial messages correctly', async () => { + // In this test, we simulate a partial message followed by a complete one. + // The first invocation of decode will throw an error (simulated incomplete message), + // and the subsequent invocation returns a valid event. + const message = { + headers: { + ':message-type': { value: 'event' }, + ':event-type': { value: 'chunk' }, + }, + body: new TextEncoder().encode( + JSON.stringify({ content: 'complete message' }), + ), + }; + + const dummyPayload1 = new Uint8Array([13, 14]); // too short, part of a frame + const frame1 = createFrame(dummyPayload1); + const dummyPayload2 = new Uint8Array([15, 16, 17, 18]); + const frame2 = createFrame(dummyPayload2); + + const mockDecode = vi + .fn() + .mockImplementationOnce(() => { + throw new Error('Incomplete data'); + }) + .mockReturnValue(message); + (EventStreamCodec as unknown as MockInstance).mockImplementation(() => ({ + decode: mockDecode, + })); + + const stream = new ReadableStream({ + start(controller) { + // Send first, incomplete frame (decode will throw error). + controller.enqueue(frame1); + // Then send a proper frame. + controller.enqueue(frame2); + controller.close(); + }, + }); + + const response = new Response(stream); + const handler = createBedrockEventStreamResponseHandler(testSchema); + const result = await handler({ + response, + url: 'test-url', + requestBodyValues: {}, + }); + + const reader = result.value.getReader(); + const { done, value } = await reader.read(); + + expect(done).toBe(false); + expect(value).toEqual({ + success: true, + value: { chunk: { content: 'complete message' } }, + rawValue: { chunk: { content: 'complete message' } }, + }); + }); +}); diff --git a/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.ts b/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.ts new file mode 100644 index 000000000000..b0497be91e87 --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-event-stream-response-handler.ts @@ -0,0 +1,104 @@ +import { EmptyResponseBodyError } from '@ai-sdk/provider'; +import { + ParseResult, + safeParseJSON, + extractResponseHeaders, + ResponseHandler, + safeValidateTypes, +} from '@ai-sdk/provider-utils'; +import { EventStreamCodec } from '@smithy/eventstream-codec'; +import { toUtf8, fromUtf8 } from '@smithy/util-utf8'; +import { ZodSchema } from 'zod'; + +// https://docs.aws.amazon.com/lexv2/latest/dg/event-stream-encoding.html +export const createBedrockEventStreamResponseHandler = + ( + chunkSchema: ZodSchema, + ): ResponseHandler>> => + async ({ response }: { response: Response }) => { + const responseHeaders = extractResponseHeaders(response); + + if (response.body == null) { + throw new EmptyResponseBodyError({}); + } + + const codec = new EventStreamCodec(toUtf8, fromUtf8); + let buffer = new Uint8Array(0); + const textDecoder = new TextDecoder(); + + return { + responseHeaders, + value: response.body.pipeThrough( + new TransformStream>({ + transform(chunk, controller) { + // Append new chunk to buffer. + const newBuffer = new Uint8Array(buffer.length + chunk.length); + newBuffer.set(buffer); + newBuffer.set(chunk, buffer.length); + buffer = newBuffer; + + // Try to decode messages from buffer. + while (buffer.length >= 4) { + // The first 4 bytes are the total length (big-endian). + const totalLength = new DataView( + buffer.buffer, + buffer.byteOffset, + buffer.byteLength, + ).getUint32(0, false); + + // If we don't have the full message yet, wait for more chunks. + if (buffer.length < totalLength) { + break; + } + + try { + // 3) Decode exactly the sub-slice for this event. + const subView = buffer.subarray(0, totalLength); + const decoded = codec.decode(subView); + + // Slice the used bytes out of the buffer, removing this message. + buffer = buffer.slice(totalLength); + + // Process the message. + if (decoded.headers[':message-type']?.value === 'event') { + const data = textDecoder.decode(decoded.body); + + // Wrap the data in the `:event-type` field to match the expected schema. + const parsedDataResult = safeParseJSON({ text: data }); + if (!parsedDataResult.success) { + controller.enqueue(parsedDataResult); + break; + } + + // The `p` field appears to be padding or some other non-functional field. + delete (parsedDataResult.value as any).p; + let wrappedData = { + [decoded.headers[':event-type']?.value as string]: + parsedDataResult.value, + }; + + // Re-validate with the expected schema. + const validatedWrappedData = safeValidateTypes({ + value: wrappedData, + schema: chunkSchema, + }); + if (!validatedWrappedData.success) { + controller.enqueue(validatedWrappedData); + } else { + controller.enqueue({ + success: true, + value: validatedWrappedData.value, + rawValue: wrappedData, + }); + } + } + } catch (e) { + // If we can't decode a complete message, wait for more data + break; + } + } + }, + }), + ), + }; + }; diff --git a/packages/amazon-bedrock/src/bedrock-prepare-tools.ts b/packages/amazon-bedrock/src/bedrock-prepare-tools.ts index ee20c2f64fde..e79d5082dfa6 100644 --- a/packages/amazon-bedrock/src/bedrock-prepare-tools.ts +++ b/packages/amazon-bedrock/src/bedrock-prepare-tools.ts @@ -4,17 +4,17 @@ import { UnsupportedFunctionalityError, } from '@ai-sdk/provider'; import { - Tool, - ToolConfiguration, - ToolInputSchema, -} from '@aws-sdk/client-bedrock-runtime'; + BedrockTool, + BedrockToolConfiguration, + BedrockToolInputSchema, +} from './bedrock-api-types'; export function prepareTools( mode: Parameters[0]['mode'] & { type: 'regular'; }, ): { - toolConfig: ToolConfiguration; // note: do not rename, name required by Bedrock + toolConfig: BedrockToolConfiguration; // note: do not rename, name required by Bedrock toolWarnings: LanguageModelV1CallWarning[]; } { // when the tools array is empty, change it to undefined to prevent errors: @@ -28,7 +28,7 @@ export function prepareTools( } const toolWarnings: LanguageModelV1CallWarning[] = []; - const bedrockTools: Tool[] = []; + const bedrockTools: BedrockTool[] = []; for (const tool of tools) { if (tool.type === 'provider-defined') { @@ -40,7 +40,7 @@ export function prepareTools( description: tool.description, inputSchema: { json: tool.parameters, - } as ToolInputSchema, + } as BedrockToolInputSchema, }, }); } diff --git a/packages/amazon-bedrock/src/bedrock-provider.test.ts b/packages/amazon-bedrock/src/bedrock-provider.test.ts new file mode 100644 index 000000000000..da357c1174ae --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-provider.test.ts @@ -0,0 +1,125 @@ +import { describe, it, expect, vi, beforeEach, Mock } from 'vitest'; +import { createAmazonBedrock } from './bedrock-provider'; +import { BedrockChatLanguageModel } from './bedrock-chat-language-model'; +import { BedrockEmbeddingModel } from './bedrock-embedding-model'; +import { loadSetting } from '@ai-sdk/provider-utils'; + +// Add type assertions for the mocked classes +const BedrockChatLanguageModelMock = + BedrockChatLanguageModel as unknown as Mock; +const BedrockEmbeddingModelMock = BedrockEmbeddingModel as unknown as Mock; + +vi.mock('./bedrock-chat-language-model', () => ({ + BedrockChatLanguageModel: vi.fn(), +})); + +vi.mock('./bedrock-embedding-model', () => ({ + BedrockEmbeddingModel: vi.fn(), +})); + +vi.mock('@ai-sdk/provider-utils', () => ({ + loadSetting: vi.fn().mockImplementation(({ settingValue }) => 'us-east-1'), + withoutTrailingSlash: vi.fn(url => url), + generateId: vi.fn().mockReturnValue('mock-id'), +})); + +describe('AmazonBedrockProvider', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('createAmazonBedrock', () => { + it('should create a provider instance with default options', () => { + const provider = createAmazonBedrock(); + const model = provider('anthropic.claude-v2'); + + const constructorCall = BedrockChatLanguageModelMock.mock.calls[0]; + expect(constructorCall[0]).toBe('anthropic.claude-v2'); + expect(constructorCall[1]).toEqual({}); + expect(constructorCall[2].headers).toEqual({}); + expect(constructorCall[2].baseUrl()).toBe( + 'https://bedrock-runtime.us-east-1.amazonaws.com', + ); + }); + + it('should create a provider instance with custom options', () => { + const customHeaders = { 'Custom-Header': 'value' }; + const options = { + region: 'eu-west-1', + baseURL: 'https://custom.url', + headers: customHeaders, + }; + + const provider = createAmazonBedrock(options); + provider('anthropic.claude-v2'); + + const constructorCall = BedrockChatLanguageModelMock.mock.calls[0]; + expect(constructorCall[2].headers).toEqual(customHeaders); + expect(constructorCall[2].baseUrl()).toBe('https://custom.url'); + }); + + it('should pass headers to embedding model', () => { + const customHeaders = { 'Custom-Header': 'value' }; + const provider = createAmazonBedrock({ + headers: customHeaders, + }); + + provider.embedding('amazon.titan-embed-text-v1'); + + const constructorCall = BedrockEmbeddingModelMock.mock.calls[0]; + expect(constructorCall[2].headers).toEqual(customHeaders); + }); + + it('should throw error when called with new keyword', () => { + const provider = createAmazonBedrock(); + expect(() => { + new (provider as any)(); + }).toThrow( + 'The Amazon Bedrock model function cannot be called with the new keyword.', + ); + }); + }); + + describe('provider methods', () => { + it('should create a chat model via function call', () => { + const provider = createAmazonBedrock(); + const modelId = 'anthropic.claude-v2'; + const settings = { additionalModelRequestFields: { foo: 'bar' } }; + + const model = provider(modelId, settings); + + const constructorCall = BedrockChatLanguageModelMock.mock.calls[0]; + expect(constructorCall[0]).toBe(modelId); + expect(constructorCall[1]).toEqual(settings); + expect(model).toBeInstanceOf(BedrockChatLanguageModel); + }); + + it('should create a chat model via languageModel method', () => { + const provider = createAmazonBedrock(); + const modelId = 'anthropic.claude-v2'; + const settings = { additionalModelRequestFields: { foo: 'bar' } }; + + const model = provider.languageModel(modelId, settings); + + const constructorCall = BedrockChatLanguageModelMock.mock.calls[0]; + expect(constructorCall[0]).toBe(modelId); + expect(constructorCall[1]).toEqual(settings); + expect(model).toBeInstanceOf(BedrockChatLanguageModel); + }); + + it('should create an embedding model', () => { + const provider = createAmazonBedrock(); + const modelId = 'amazon.titan-embed-text-v1'; + + const model = provider.embedding(modelId, { + dimensions: 1024, + normalize: true, + }); + + const constructorCall = BedrockEmbeddingModelMock.mock.calls[0]; + expect(constructorCall[0]).toBe(modelId); + expect(constructorCall[1]).toEqual({ dimensions: 1024, normalize: true }); + expect(model).toBeInstanceOf(BedrockEmbeddingModel); + }); + }); +}); diff --git a/packages/amazon-bedrock/src/bedrock-provider.ts b/packages/amazon-bedrock/src/bedrock-provider.ts index a8e697bd50b6..7bfb4c000fe3 100644 --- a/packages/amazon-bedrock/src/bedrock-provider.ts +++ b/packages/amazon-bedrock/src/bedrock-provider.ts @@ -4,14 +4,11 @@ import { ProviderV1, } from '@ai-sdk/provider'; import { + FetchFunction, generateId, - loadOptionalSetting, loadSetting, + withoutTrailingSlash, } from '@ai-sdk/provider-utils'; -import { - BedrockRuntimeClient, - BedrockRuntimeClientConfig, -} from '@aws-sdk/client-bedrock-runtime'; import { BedrockChatLanguageModel } from './bedrock-chat-language-model'; import { BedrockChatModelId, @@ -22,19 +19,47 @@ import { BedrockEmbeddingModelId, BedrockEmbeddingSettings, } from './bedrock-embedding-settings'; +import { createSigV4FetchFunction } from './bedrock-sigv4-fetch'; export interface AmazonBedrockProviderSettings { + /** +The AWS region to use for the Bedrock provider. Defaults to the value of the +`AWS_REGION` environment variable. + */ region?: string; + + /** +The AWS access key ID to use for the Bedrock provider. Defaults to the value of the + */ accessKeyId?: string; + + /** +The AWS secret access key to use for the Bedrock provider. Defaults to the value of the +`AWS_SECRET_ACCESS_KEY` environment variable. + */ secretAccessKey?: string; + + /** +The AWS session token to use for the Bedrock provider. Defaults to the value of the +`AWS_SESSION_TOKEN` environment variable. + */ sessionToken?: string; /** - * Complete Bedrock configuration for setting advanced authentication and - * other options. When this is provided, the region, accessKeyId, and - * secretAccessKey settings are ignored. +Base URL for the Bedrock API calls. + */ + baseURL?: string; + + /** +Custom headers to include in the requests. */ - bedrockOptions?: BedrockRuntimeClientConfig; + headers?: Record; + + /** +Custom fetch implementation. You can use it as a middleware to intercept requests, +or to provide a custom fetch implementation for e.g. testing. +*/ + fetch?: FetchFunction; // for testing generateId?: () => string; @@ -63,42 +88,34 @@ Create an Amazon Bedrock provider instance. export function createAmazonBedrock( options: AmazonBedrockProviderSettings = {}, ): AmazonBedrockProvider { - const createBedrockRuntimeClient = () => - new BedrockRuntimeClient( - options.bedrockOptions ?? { - region: loadSetting({ + const sigv4Fetch = createSigV4FetchFunction( + { + region: options.region, + accessKeyId: options.accessKeyId, + secretAccessKey: options.secretAccessKey, + sessionToken: options.sessionToken, + }, + options.fetch, + ); + const getBaseUrl = (): string => + withoutTrailingSlash( + options.baseURL ?? + `https://bedrock-runtime.${loadSetting({ settingValue: options.region, settingName: 'region', environmentVariableName: 'AWS_REGION', description: 'AWS region', - }), - credentials: { - accessKeyId: loadSetting({ - settingValue: options.accessKeyId, - settingName: 'accessKeyId', - environmentVariableName: 'AWS_ACCESS_KEY_ID', - description: 'AWS access key ID', - }), - secretAccessKey: loadSetting({ - settingValue: options.secretAccessKey, - settingName: 'secretAccessKey', - environmentVariableName: 'AWS_SECRET_ACCESS_KEY', - description: 'AWS secret access key', - }), - sessionToken: loadOptionalSetting({ - settingValue: options.sessionToken, - environmentVariableName: 'AWS_SESSION_TOKEN', - }), - }, - }, - ); + })}.amazonaws.com`, + ) ?? `https://bedrock-runtime.us-east-1.amazonaws.com`; const createChatModel = ( modelId: BedrockChatModelId, settings: BedrockChatSettings = {}, ) => new BedrockChatLanguageModel(modelId, settings, { - client: createBedrockRuntimeClient(), + baseUrl: getBaseUrl, + headers: options.headers ?? {}, + fetch: sigv4Fetch, generateId, }); @@ -120,7 +137,9 @@ export function createAmazonBedrock( settings: BedrockEmbeddingSettings = {}, ) => new BedrockEmbeddingModel(modelId, settings, { - client: createBedrockRuntimeClient(), + baseUrl: getBaseUrl, + headers: options.headers ?? {}, + fetch: sigv4Fetch, }); provider.languageModel = createChatModel; diff --git a/packages/amazon-bedrock/src/bedrock-sigv4-fetch.test.ts b/packages/amazon-bedrock/src/bedrock-sigv4-fetch.test.ts new file mode 100644 index 000000000000..e283e947b6fc --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-sigv4-fetch.test.ts @@ -0,0 +1,244 @@ +import { createSigV4FetchFunction } from './bedrock-sigv4-fetch'; +import { vi, describe, it, expect, afterEach } from 'vitest'; + +// Mock AwsV4Signer so that no real crypto calls are made. +vi.mock('aws4fetch', () => { + class MockAwsV4Signer { + options: any; + constructor(options: any) { + this.options = options; + } + async sign() { + // Return a fake Headers instance with predetermined signing headers. + const headers = new Headers(); + headers.set('x-amz-date', '20240315T000000Z'); + headers.set('authorization', 'AWS4-HMAC-SHA256 Credential=test'); + if (this.options.sessionToken) { + headers.set('x-amz-security-token', this.options.sessionToken); + } + return { headers }; + } + } + return { AwsV4Signer: MockAwsV4Signer }; +}); + +describe('createSigV4FetchFunction', () => { + afterEach(() => { + vi.restoreAllMocks(); + }); + + it('should bypass signing for non-POST requests', async () => { + const dummyResponse = new Response('OK', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const fetchFn = createSigV4FetchFunction({}, dummyFetch); + const response = await fetchFn('http://example.com', { method: 'GET' }); + expect(dummyFetch).toHaveBeenCalledWith('http://example.com', { + method: 'GET', + }); + expect(response).toBe(dummyResponse); + }); + + it('should bypass signing if POST request has no body', async () => { + const dummyResponse = new Response('OK', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const fetchFn = createSigV4FetchFunction({}, dummyFetch); + const response = await fetchFn('http://example.com', { method: 'POST' }); + expect(dummyFetch).toHaveBeenCalledWith('http://example.com', { + method: 'POST', + }); + expect(response).toBe(dummyResponse); + }); + + it('should handle a POST request with a string body and merge signed headers', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + // Provide settings (including a sessionToken) so that the signer includes that header. + const settings = { + region: 'us-west-2', + accessKeyId: 'test-access-key', + secretAccessKey: 'test-secret', + sessionToken: 'test-session-token', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const inputUrl = 'http://example.com'; + const init: RequestInit = { + method: 'POST', + body: '{"test": "data"}', + headers: { + 'Content-Type': 'application/json', + 'Custom-Header': 'value', + }, + }; + + await fetchFn(inputUrl, init); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + // `combinedHeaders` should merge the original headers with the signing + // headers added by the AwsV4Signer mock. + const headers = calledInit.headers as Record; + expect(headers['Content-Type']).toEqual('application/json'); + expect(headers['Custom-Header']).toEqual('value'); + expect(headers['Empty-Header']).toBeUndefined(); + expect(headers['x-amz-date']).toEqual('20240315T000000Z'); + expect(headers['authorization']).toEqual( + 'AWS4-HMAC-SHA256 Credential=test', + ); + expect(headers['x-amz-security-token']).toEqual('test-session-token'); + // Body is left unmodified for a string body. + expect(calledInit.body).toEqual('{"test": "data"}'); + }); + + it('should handle non-string body by stringifying it', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const settings = { + region: 'us-west-2', + accessKeyId: 'key', + secretAccessKey: 'secret', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const inputUrl = 'http://example.com'; + const jsonBody = { field: 'value' }; + + await fetchFn(inputUrl, { + method: 'POST', + body: jsonBody as unknown as BodyInit, + headers: {}, + }); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + // The body should be stringified. + expect(calledInit.body).toEqual(JSON.stringify(jsonBody)); + }); + + it('should handle Uint8Array body', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const settings = { + region: 'us-west-2', + accessKeyId: 'key', + secretAccessKey: 'secret', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const inputUrl = 'http://example.com'; + const uint8Body = new TextEncoder().encode('binaryTest'); + + await fetchFn(inputUrl, { + method: 'POST', + body: uint8Body, + headers: {}, + }); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + // The Uint8Array body should have been decoded to a string. + expect(calledInit.body).toEqual('binaryTest'); + }); + + it('should handle ArrayBuffer body', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const settings = { + region: 'us-west-2', + accessKeyId: 'key', + secretAccessKey: 'secret', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const inputUrl = 'http://example.com'; + const text = 'bufferTest'; + const buffer = new TextEncoder().encode(text).buffer; + + await fetchFn(inputUrl, { + method: 'POST', + body: buffer, + headers: {}, + }); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + expect(calledInit.body).toEqual(text); + }); + + it('should extract headers from a Headers instance', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const settings = { + region: 'us-west-2', + accessKeyId: 'key', + secretAccessKey: 'secret', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const h = new Headers(); + h.set('A', 'value-a'); + h.set('B', 'value-b'); + + await fetchFn('http://example.com', { + method: 'POST', + body: '{"test": "data"}', + headers: h, + }); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + const headers = calledInit.headers as Record; + // Depending on the runtime, header keys might be normalized (typically lowercased). + expect(headers['a'] || headers['A']).toEqual('value-a'); + expect(headers['b'] || headers['B']).toEqual('value-b'); + }); + + it('should handle headers provided as an array', async () => { + const dummyResponse = new Response('Signed', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const settings = { + region: 'us-west-2', + accessKeyId: 'key', + secretAccessKey: 'secret', + }; + const fetchFn = createSigV4FetchFunction(settings, dummyFetch); + + const headersArray: [string, string][] = [ + ['Array-Header', 'array-value'], + ['Another-Header', 'another-value'], + ]; + + await fetchFn('http://example.com', { + method: 'POST', + body: '{"test": "data"}', + headers: headersArray, + }); + expect(dummyFetch).toHaveBeenCalled(); + const calledInit = dummyFetch.mock.calls[0][1] as RequestInit; + const headers = calledInit.headers as Record; + expect(headers['array-header'] || headers['Array-Header']).toEqual( + 'array-value', + ); + expect(headers['another-header'] || headers['Another-Header']).toEqual( + 'another-value', + ); + // Also check that the signing headers are included. + expect(headers['x-amz-date']).toEqual('20240315T000000Z'); + expect(headers['authorization']).toEqual( + 'AWS4-HMAC-SHA256 Credential=test', + ); + }); + + it('should call original fetch if init is undefined', async () => { + const dummyResponse = new Response('OK', { status: 200 }); + const dummyFetch = vi.fn().mockResolvedValue(dummyResponse); + + const fetchFn = createSigV4FetchFunction({}, dummyFetch); + const response = await fetchFn('http://example.com'); + expect(dummyFetch).toHaveBeenCalledWith('http://example.com', undefined); + expect(response).toBe(dummyResponse); + }); +}); diff --git a/packages/amazon-bedrock/src/bedrock-sigv4-fetch.ts b/packages/amazon-bedrock/src/bedrock-sigv4-fetch.ts new file mode 100644 index 000000000000..d3f1002308f3 --- /dev/null +++ b/packages/amazon-bedrock/src/bedrock-sigv4-fetch.ts @@ -0,0 +1,149 @@ +import { + FetchFunction, + loadOptionalSetting, + loadSetting, + combineHeaders, +} from '@ai-sdk/provider-utils'; +import { AwsV4Signer } from 'aws4fetch'; + +export interface SigV4Settings { + region?: string; + accessKeyId?: string; + secretAccessKey?: string; + sessionToken?: string; +} + +/** + * Creates a fetch function that applies AWS Signature Version 4 signing. + * + * This wrapper inspects the RequestInit and, if it is a POST with a body, it uses + * AwsV4Signer to add the required signing headers. It ensures that if the request body + * is already stringified it will be reused directly—saving us from having to call JSON.stringify + * again on a large payload. + * + * @param settings - Settings to use when signing (region, access key, secret, etc.). + * @param originalFetch - Optional original fetch implementation to wrap. Defaults to global fetch. + * @returns A FetchFunction that signs requests before passing them to the underlying fetch. + */ +export function createSigV4FetchFunction( + settings: SigV4Settings = {}, + originalFetch?: FetchFunction, +): FetchFunction { + const fetchImpl = originalFetch || globalThis.fetch; + return async ( + input: RequestInfo | URL, + init?: RequestInit, + ): Promise => { + // We only need to sign POST requests that have a body. + if (!init || init.method?.toUpperCase() !== 'POST' || !init.body) { + return fetchImpl(input, init); + } + + // Determine the URL from the fetch input. + const url = + typeof input === 'string' + ? input + : input instanceof URL + ? input.href + : input.url; + + // Extract headers from the RequestInit. + let originalHeaders: Record = {}; + if (init.headers) { + if (init.headers instanceof Headers) { + originalHeaders = convertHeadersToRecord(init.headers); + } else if (Array.isArray(init.headers)) { + for (const [k, v] of init.headers) { + originalHeaders[k] = v; + } + } else { + originalHeaders = { ...init.headers } as Record; + } + } + + // Prepare the body as a string. + // If the body is already a string, do not re-stringify. + let bodyString: string; + if (typeof init.body === 'string') { + bodyString = init.body; + } else if (init.body instanceof Uint8Array) { + bodyString = new TextDecoder().decode(init.body); + } else if (init.body instanceof ArrayBuffer) { + bodyString = new TextDecoder().decode(new Uint8Array(init.body)); + } else { + // Fallback: assume it's a plain object. + bodyString = JSON.stringify(init.body); + } + + // Resolve AWS credentials and region from settings and environment variables. + // TODO: Build credentials set earlier in provider control flow and pass in here. + const region = loadSetting({ + settingValue: settings.region, + settingName: 'region', + environmentVariableName: 'AWS_REGION', + description: 'AWS region', + }); + const accessKeyId = loadSetting({ + settingValue: settings.accessKeyId, + settingName: 'accessKeyId', + environmentVariableName: 'AWS_ACCESS_KEY_ID', + description: 'AWS access key ID', + }); + const secretAccessKey = loadSetting({ + settingValue: settings.secretAccessKey, + settingName: 'secretAccessKey', + environmentVariableName: 'AWS_SECRET_ACCESS_KEY', + description: 'AWS secret access key', + }); + const sessionToken = loadOptionalSetting({ + settingValue: settings.sessionToken, + environmentVariableName: 'AWS_SESSION_TOKEN', + }); + + // Create the signer, passing the already stringified body. + const signer = new AwsV4Signer({ + url, + method: 'POST', + headers: Object.entries(removeUndefinedEntries(originalHeaders)), + body: bodyString, + region, + accessKeyId, + secretAccessKey, + ...(sessionToken && { sessionToken }), + service: 'bedrock', + }); + + const result = await signer.sign(); + const signedHeaders = convertHeadersToRecord(result.headers); + const mergedHeaders = removeUndefinedEntries( + combineHeaders(originalHeaders, signedHeaders), + ); + + // Create a new RequestInit with the clean headers. + const newInit: RequestInit = { + ...init, + body: bodyString, + headers: mergedHeaders, + }; + + // Invoke the underlying fetch implementation with the new headers. + return fetchImpl(input, newInit); + }; +} + +function convertHeadersToRecord(headers: Headers): Record { + const record: Record = {}; + headers.forEach((value, key) => { + record[key] = value; + }); + return record; +} + +// TODO: export from provider-utils. +function removeUndefinedEntries( + record: Record, +): Record { + return Object.fromEntries( + Object.entries(record).filter(([_key, value]) => value != null), + ) as Record; +} diff --git a/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.test.ts b/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.test.ts index 90397677c48e..10a115ef4e85 100644 --- a/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.test.ts +++ b/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.test.ts @@ -51,14 +51,16 @@ describe('user messages', () => { { image: { format: 'png', - source: { bytes: new Uint8Array([0, 1, 2, 3]) }, + source: { bytes: 'AAECAw==' }, }, }, { document: { format: 'pdf', name: expect.any(String), - source: { bytes: Buffer.from(fileData) }, + source: { + bytes: 'AAECAw==', + }, }, }, ], diff --git a/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.ts b/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.ts index 4e4cc47bb73f..7520e40b8e3a 100644 --- a/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.ts +++ b/packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.ts @@ -4,7 +4,7 @@ import { UnsupportedFunctionalityError, } from '@ai-sdk/provider'; import { createIdGenerator } from '@ai-sdk/provider-utils'; -import { DocumentFormat, ImageFormat } from '@aws-sdk/client-bedrock-runtime'; +import { BedrockDocumentFormat, BedrockImageFormat } from './bedrock-api-types'; import { BedrockAssistantMessage, BedrockMessagesPrompt, @@ -67,9 +67,13 @@ export function convertToBedrockChatMessages( bedrockContent.push({ image: { - format: part.mimeType?.split('/')?.[1] as ImageFormat, + format: part.mimeType?.split( + '/', + )?.[1] as BedrockImageFormat, source: { - bytes: part.image ?? (part.image as Uint8Array), + bytes: Buffer.from( + part.image ?? (part.image as Uint8Array), + ).toString('base64'), }, }, }); @@ -88,10 +92,10 @@ export function convertToBedrockChatMessages( document: { format: part.mimeType?.split( '/', - )?.[1] as DocumentFormat, + )?.[1] as BedrockDocumentFormat, name: generateFileId(), source: { - bytes: Buffer.from(part.data, 'base64'), + bytes: part.data, }, }, }); diff --git a/packages/amazon-bedrock/src/map-bedrock-finish-reason.ts b/packages/amazon-bedrock/src/map-bedrock-finish-reason.ts index 2d8d68540b32..b10310efc05b 100644 --- a/packages/amazon-bedrock/src/map-bedrock-finish-reason.ts +++ b/packages/amazon-bedrock/src/map-bedrock-finish-reason.ts @@ -1,8 +1,8 @@ import { LanguageModelV1FinishReason } from '@ai-sdk/provider'; -import { StopReason } from '@aws-sdk/client-bedrock-runtime'; +import { BedrockStopReason } from './bedrock-api-types'; export function mapBedrockFinishReason( - finishReason?: StopReason, + finishReason?: BedrockStopReason, ): LanguageModelV1FinishReason { switch (finishReason) { case 'stop_sequence': diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index a0d9eb21ce84..04a71716cd18 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -449,7 +449,7 @@ importers: version: link:../../packages/ai langchain: specifier: 0.1.36 - version: 0.1.36(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(fast-xml-parser@4.4.1)(ignore@5.3.2)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(playwright@1.46.0)(ws@8.18.0) + version: 0.1.36(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(fast-xml-parser@4.4.1)(ignore@5.3.2)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(playwright@1.46.0)(ws@8.18.0) next: specifier: latest version: 15.1.6(@opentelemetry/api@1.9.0)(@playwright/test@1.46.0)(react-dom@18.2.0(react@18.2.0))(react@18.2.0) @@ -1174,22 +1174,22 @@ importers: '@ai-sdk/provider-utils': specifier: 2.1.6 version: link:../provider-utils - '@aws-sdk/client-bedrock-runtime': - specifier: ^3.663.0 - version: 3.663.0 + '@smithy/eventstream-codec': + specifier: ^4.0.1 + version: 4.0.1 + '@smithy/util-utf8': + specifier: ^4.0.0 + version: 4.0.0 + aws4fetch: + specifier: ^1.0.20 + version: 1.0.20 devDependencies: - '@smithy/types': - specifier: ^3.5.0 - version: 3.5.0 '@types/node': specifier: ^18.19.54 version: 18.19.54 '@vercel/ai-tsconfig': specifier: workspace:* version: link:../../tools/tsconfig - aws-sdk-client-mock: - specifier: ^4.0.2 - version: 4.0.2 tsup: specifier: ^8.3.0 version: 8.3.0(jiti@2.4.0)(postcss@8.4.49)(tsx@4.19.2)(typescript@5.6.3)(yaml@2.5.0) @@ -6592,18 +6592,6 @@ packages: '@sinonjs/fake-timers@10.3.0': resolution: {integrity: sha512-V4BG07kuYSUkTCSBHG8G8TNhM+F19jXFWnQtzj+we8DrkpSBCee9Z3Ms8yiGer/dlmhe35/Xdgyo3/0rQKg7YA==} - '@sinonjs/fake-timers@11.2.2': - resolution: {integrity: sha512-G2piCSxQ7oWOxwGSAyFHfPIsyeJGXYtc6mFbnFA+kRXkiEnTl8c/8jul2S329iFBnDI9HGoeWWAZvuvOkZccgw==} - - '@sinonjs/fake-timers@13.0.2': - resolution: {integrity: sha512-4Bb+oqXZTSTZ1q27Izly9lv8B9dlV61CROxPiVtywwzv5SnytJqhvYe6FclHYuXml4cd1VHPo1zd5PmTeJozvA==} - - '@sinonjs/samsam@8.0.2': - resolution: {integrity: sha512-v46t/fwnhejRSFTGqbpn9u+LQ9xJDse10gNnPgAcxgdoCDMXj/G2asWAC/8Qs+BAZDicX+MNZouXT1A7c83kVw==} - - '@sinonjs/text-encoding@0.7.3': - resolution: {integrity: sha512-DE427ROAphMQzU4ENbliGYrBSYPXF+TtLg9S8vzeA+OF4ZKzoDdzfL8sxuMUGS/lgRhM6j1URSk9ghf7Xo1tyA==} - '@smithy/abort-controller@3.1.5': resolution: {integrity: sha512-DhNPnqTqPoG8aZ5dWkFOgsuY+i0GQ3CI6hMmvCoduNsnU9gUZWZBwGfDQsTTB7NvFPkom1df7jMIJWU90kuXXg==} engines: {node: '>=16.0.0'} @@ -6623,6 +6611,10 @@ packages: '@smithy/eventstream-codec@3.1.6': resolution: {integrity: sha512-SBiOYPBH+5wOyPS7lfI150ePfGLhnp/eTu5RnV9xvhGvRiKfnl6HzRK9wehBph+il8FxS9KTeadx7Rcmf1GLPQ==} + '@smithy/eventstream-codec@4.0.1': + resolution: {integrity: sha512-Q2bCAAR6zXNVtJgifsU16ZjKGqdw/DyecKNgIgi7dlqw04fqDu0mnq+JmGphqheypVc64CYq3azSuCpAdFk2+A==} + engines: {node: '>=18.0.0'} + '@smithy/eventstream-serde-browser@3.0.10': resolution: {integrity: sha512-1i9aMY6Pl/SmA6NjvidxnfBLHMPzhKu2BP148pEt5VwhMdmXn36PE2kWKGa9Hj8b0XGtCTRucpCncylevCtI7g==} engines: {node: '>=16.0.0'} @@ -6657,6 +6649,10 @@ packages: resolution: {integrity: sha512-+Fsu6Q6C4RSJiy81Y8eApjEB5gVtM+oFKTffg+jSuwtvomJJrhUJBu2zS8wjXSgH/g1MKEWrzyChTBe6clb5FQ==} engines: {node: '>=16.0.0'} + '@smithy/is-array-buffer@4.0.0': + resolution: {integrity: sha512-saYhF8ZZNoJDTvJBEWgeBccCg+yvp1CX+ed12yORU3NilJScfc6gfch2oVb4QgxZrGUx3/ZJlb+c/dJbyupxlw==} + engines: {node: '>=18.0.0'} + '@smithy/middleware-content-length@3.0.9': resolution: {integrity: sha512-t97PidoGElF9hTtLCrof32wfWMqC5g2SEJNxaVH3NjlatuNGsdxXRYO/t+RPnxA15RpYiS0f+zG7FuE2DeGgjA==} engines: {node: '>=16.0.0'} @@ -6721,6 +6717,10 @@ packages: resolution: {integrity: sha512-QN0twHNfe8mNJdH9unwsCK13GURU7oEAZqkBI+rsvpv1jrmserO+WnLE7jidR9W/1dxwZ0u/CB01mV2Gms/K2Q==} engines: {node: '>=16.0.0'} + '@smithy/types@4.1.0': + resolution: {integrity: sha512-enhjdwp4D7CXmwLtD6zbcDMbo6/T6WtuuKCY49Xxc6OMOmUWlBEBDREsxxgV2LIdeQPW756+f97GzcgAwp3iLw==} + engines: {node: '>=18.0.0'} + '@smithy/url-parser@3.0.7': resolution: {integrity: sha512-70UbSSR8J97c1rHZOWhl+VKiZDqHWxs/iW8ZHrHp5fCCPLSBE7GcUlUvKSle3Ca+J9LLbYCj/A79BxztBvAfpA==} @@ -6743,6 +6743,10 @@ packages: resolution: {integrity: sha512-aEOHCgq5RWFbP+UDPvPot26EJHjOC+bRgse5A8V3FSShqd5E5UN4qc7zkwsvJPPAVsf73QwYcHN1/gt/rtLwQA==} engines: {node: '>=16.0.0'} + '@smithy/util-buffer-from@4.0.0': + resolution: {integrity: sha512-9TOQ7781sZvddgO8nxueKi3+yGvkY35kotA0Y6BWRajAv8jjmigQ1sBwz0UX47pQMYXJPahSKEKYFgt+rXdcug==} + engines: {node: '>=18.0.0'} + '@smithy/util-config-provider@3.0.0': resolution: {integrity: sha512-pbjk4s0fwq3Di/ANL+rCvJMKM5bzAQdE5S/6RL5NXgMExFAi6UgQMPOm5yPaIWPpr+EOXKXRonJ3FoxKf4mCJQ==} engines: {node: '>=16.0.0'} @@ -6763,6 +6767,10 @@ packages: resolution: {integrity: sha512-eFndh1WEK5YMUYvy3lPlVmYY/fZcQE1D8oSf41Id2vCeIkKJXPcYDCZD+4+xViI6b1XSd7tE+s5AmXzz5ilabQ==} engines: {node: '>=16.0.0'} + '@smithy/util-hex-encoding@4.0.0': + resolution: {integrity: sha512-Yk5mLhHtfIgW2W2WQZWSg5kuMZCVbvhFmC7rV4IO2QqnZdbEFPmQnCcGMAX2z/8Qj3B9hYYNjZOhWym+RwhePw==} + engines: {node: '>=18.0.0'} + '@smithy/util-middleware@3.0.7': resolution: {integrity: sha512-OVA6fv/3o7TMJTpTgOi1H5OTwnuUa8hzRzhSFDtZyNxi6OZ70L/FHattSmhE212I7b6WSOJAAmbYnvcjTHOJCA==} engines: {node: '>=16.0.0'} @@ -6787,6 +6795,10 @@ packages: resolution: {integrity: sha512-rUeT12bxFnplYDe815GXbq/oixEGHfRFFtcTF3YdDi/JaENIM6aSYYLJydG83UNzLXeRI5K8abYd/8Sp/QM0kA==} engines: {node: '>=16.0.0'} + '@smithy/util-utf8@4.0.0': + resolution: {integrity: sha512-b+zebfKCfRdgNJDknHCob3O7FpeYQN6ZG6YLExMcasDHsCXlsXCEuiPZeLnJLpwa5dvPetGlnGCiMHuLwGvFow==} + engines: {node: '>=18.0.0'} + '@solid-primitives/trigger@1.1.0': resolution: {integrity: sha512-00BbAiXV66WwjHuKZc3wr0+GLb9C24mMUmi3JdTpNFgHBbrQGrIHubmZDg36c5/7wH+E0GQtOOanwQS063PO+A==} peerDependencies: @@ -7134,12 +7146,6 @@ packages: '@types/shimmer@1.2.0': resolution: {integrity: sha512-UE7oxhQLLd9gub6JKIAhDq06T0F6FnztwMNRvYgjeQSBeMc1ZG/tA47EwfduvkuQS8apbkM/lpLpWsaCeYsXVg==} - '@types/sinon@17.0.3': - resolution: {integrity: sha512-j3uovdn8ewky9kRBG19bOwaZbexJu/XjtkHyjvUgt4xfPFz18dcORIMqnYh66Fx3Powhcr85NT5+er3+oViapw==} - - '@types/sinonjs__fake-timers@8.1.5': - resolution: {integrity: sha512-mQkU2jY8jJEF7YHjHvsQO8+3ughTL1mcnn96igfhONmR+fUPSKIkefQYpSe8bsly2Ep7oQbn/6VG5/9/0qcArQ==} - '@types/stack-utils@2.0.3': resolution: {integrity: sha512-9aEbYZ3TbYMznPdcdr3SmIrLXwC/AKZXQeCf9Pgao5CKb8CyHuEX5jzWPTkvregvhRJHcpRO6BFoGW9ycaOkYw==} @@ -7904,8 +7910,8 @@ packages: avvio@9.0.0: resolution: {integrity: sha512-UbYrOXgE/I+knFG+3kJr9AgC7uNo8DG+FGGODpH9Bj1O1kL/QDjBXnTem9leD3VdQKtaHjV3O85DQ7hHh4IIHw==} - aws-sdk-client-mock@4.0.2: - resolution: {integrity: sha512-saFLXQPqHuMH0A1peNIGoAFEq9B0bpS5y5qrr+Y5F86MasVkCctggHKhHPRVjGr852Nz7cLg/PBxKs6lQoK3mg==} + aws4fetch@1.0.20: + resolution: {integrity: sha512-/djoAN709iY65ETD6LKCtyyEI04XIBP5xVvfmNxsEP0uJB5tyaGBztSryRr4HqMStr9R06PisQE7m9zDTXKu6g==} axe-core@4.10.0: resolution: {integrity: sha512-Mr2ZakwQ7XUAjp7pAwQWRhhK8mQQ6JAaNWSjmjxil0R8BPioMtQsTLOolGYkji1rcL++3dCqZA3zWqpT+9Ew6g==} @@ -8851,10 +8857,6 @@ packages: resolution: {integrity: sha512-58lmxKSA4BNyLz+HHMUzlOEpg09FV+ev6ZMe3vJihgdxzgcwZ8VoEEPmALCZG9LmqfVoNMMKpttIYTVG6uDY7A==} engines: {node: '>=0.3.1'} - diff@5.2.0: - resolution: {integrity: sha512-uIFDxqpRZGZ6ThOk84hEfqWoHx2devRFvpTZcTHur85vImfaxUbTW9Ryh4CpCuDnToOP1CEtXKIgytHBPVff5A==} - engines: {node: '>=0.3.1'} - diff@7.0.0: resolution: {integrity: sha512-PJWHUb1RFevKCwaFA9RlG5tCd+FO5iRh9A8HEtkmBH2Li03iJriB6m6JIN4rGz3K3JLawI7/veA1xzRKP6ISBw==} engines: {node: '>=0.3.1'} @@ -10645,9 +10647,6 @@ packages: resolution: {integrity: sha512-ZZow9HBI5O6EPgSJLUb8n2NKgmVWTwCvHGwFuJlMjvLFqlGG6pjirPhtdsseaLZjSibD8eegzmYpUZwoIlj2cQ==} engines: {node: '>=4.0'} - just-extend@6.2.0: - resolution: {integrity: sha512-cYofQu2Xpom82S6qD778jBDpwvvy39s1l/hrYij2u9AMdQcGRpaBu6kY4mVhuno5kJVi1DAz4aiphA2WI1/OAw==} - jwa@2.0.0: resolution: {integrity: sha512-jrZ2Qx916EA+fq9cEAeCROWPTfCwi1IVHqT2tapuqLEVVDKFDENFw1oL+MwrTvH6msKxsd1YTDVw6uKEcsrLEA==} @@ -10960,10 +10959,6 @@ packages: lodash.defaults@4.2.0: resolution: {integrity: sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==} - lodash.get@4.4.2: - resolution: {integrity: sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==} - deprecated: This package is deprecated. Use the optional chaining (?.) operator instead. - lodash.isarguments@3.1.0: resolution: {integrity: sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==} @@ -11510,9 +11505,6 @@ packages: sass: optional: true - nise@6.1.1: - resolution: {integrity: sha512-aMSAzLVY7LyeM60gvBS423nBmIPP+Wy7St7hsb+8/fc1HmeoHJfLO8CKse4u3BtOZvQLJghYPI2i/1WZrEj5/g==} - nitropack@2.10.4: resolution: {integrity: sha512-sJiG/MIQlZCVSw2cQrFG1H6mLeSqHlYfFerRjLKz69vUfdu0EL2l0WdOxlQbzJr3mMv/l4cOlCCLzVRzjzzF/g==} engines: {node: ^16.11.0 || >=17.0.0} @@ -12948,9 +12940,6 @@ packages: simple-swizzle@0.2.2: resolution: {integrity: sha512-JA//kQgZtbuY83m+xT+tXJkmJncGMTFT+C+g2h2R9uxkYIrE2yy9sgmcLhCnw57/WSD+Eh3J97FPEDFnbXnDUg==} - sinon@18.0.1: - resolution: {integrity: sha512-a2N2TDY1uGviajJ6r4D1CyRAkzE9NNVlYOV1wX5xQDuAk0ONgzgRl0EjCQuRCPxOwp13ghsMwt9Gdldujs39qw==} - sirv@2.0.4: resolution: {integrity: sha512-94Bdh3cC2PKrbgSOUqTiGPWVZeSiXfKOVZNJniWoqrWrRkB1CJzBU3NEbiTsPcYy1lDsANA/THzS+9WBiy5nfQ==} engines: {node: '>= 10'} @@ -13713,10 +13702,6 @@ packages: resolution: {integrity: sha512-0fr/mIH1dlO+x7TlcMy+bIDqKPsw/70tVyeHW787goQjhmqaZe10uwLujubK9q9Lg6Fiho1KUKDYz0Z7k7g5/g==} engines: {node: '>=4'} - type-detect@4.1.0: - resolution: {integrity: sha512-Acylog8/luQ8L7il+geoSxhEkazvkslg7PSNKOX59mbB9cOveP5aq9h74Y7YU8yDpJwetzQQrfIwtf4Wp4LKcw==} - engines: {node: '>=4'} - type-fest@0.20.2: resolution: {integrity: sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ==} engines: {node: '>=10'} @@ -14604,16 +14589,19 @@ snapshots: '@aws-sdk/util-locate-window': 3.568.0 '@smithy/util-utf8': 2.3.0 tslib: 2.8.1 + optional: true '@aws-crypto/sha256-js@5.2.0': dependencies: '@aws-crypto/util': 5.2.0 '@aws-sdk/types': 3.662.0 tslib: 2.8.1 + optional: true '@aws-crypto/supports-web-crypto@5.2.0': dependencies: tslib: 2.8.1 + optional: true '@aws-crypto/util@5.2.0': dependencies: @@ -14667,9 +14655,10 @@ snapshots: '@smithy/util-retry': 3.0.7 '@smithy/util-stream': 3.1.9 '@smithy/util-utf8': 3.0.0 - tslib: 2.7.0 + tslib: 2.8.1 transitivePeerDependencies: - aws-crt + optional: true '@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0)': dependencies: @@ -14715,6 +14704,7 @@ snapshots: tslib: 2.8.1 transitivePeerDependencies: - aws-crt + optional: true '@aws-sdk/client-sso@3.662.0': dependencies: @@ -14758,6 +14748,7 @@ snapshots: tslib: 2.8.1 transitivePeerDependencies: - aws-crt + optional: true '@aws-sdk/client-sts@3.662.0': dependencies: @@ -14803,6 +14794,7 @@ snapshots: tslib: 2.8.1 transitivePeerDependencies: - aws-crt + optional: true '@aws-sdk/core@3.662.0': dependencies: @@ -14816,6 +14808,7 @@ snapshots: '@smithy/util-middleware': 3.0.7 fast-xml-parser: 4.4.1 tslib: 2.8.1 + optional: true '@aws-sdk/credential-provider-env@3.662.0': dependencies: @@ -14823,6 +14816,7 @@ snapshots: '@smithy/property-provider': 3.1.7 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/credential-provider-http@3.662.0': dependencies: @@ -14835,6 +14829,7 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/util-stream': 3.1.9 tslib: 2.8.1 + optional: true '@aws-sdk/credential-provider-ini@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0)': dependencies: @@ -14853,6 +14848,7 @@ snapshots: transitivePeerDependencies: - '@aws-sdk/client-sso-oidc' - aws-crt + optional: true '@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0)': dependencies: @@ -14872,6 +14868,7 @@ snapshots: - '@aws-sdk/client-sso-oidc' - '@aws-sdk/client-sts' - aws-crt + optional: true '@aws-sdk/credential-provider-process@3.662.0': dependencies: @@ -14880,6 +14877,7 @@ snapshots: '@smithy/shared-ini-file-loader': 3.1.8 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/credential-provider-sso@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))': dependencies: @@ -14893,6 +14891,7 @@ snapshots: transitivePeerDependencies: - '@aws-sdk/client-sso-oidc' - aws-crt + optional: true '@aws-sdk/credential-provider-web-identity@3.662.0(@aws-sdk/client-sts@3.662.0)': dependencies: @@ -14901,6 +14900,7 @@ snapshots: '@smithy/property-provider': 3.1.7 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/middleware-host-header@3.662.0': dependencies: @@ -14908,12 +14908,14 @@ snapshots: '@smithy/protocol-http': 4.1.4 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/middleware-logger@3.662.0': dependencies: '@aws-sdk/types': 3.662.0 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/middleware-recursion-detection@3.662.0': dependencies: @@ -14921,6 +14923,7 @@ snapshots: '@smithy/protocol-http': 4.1.4 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/middleware-user-agent@3.662.0': dependencies: @@ -14929,6 +14932,7 @@ snapshots: '@smithy/protocol-http': 4.1.4 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/region-config-resolver@3.662.0': dependencies: @@ -14938,6 +14942,7 @@ snapshots: '@smithy/util-config-provider': 3.0.0 '@smithy/util-middleware': 3.0.7 tslib: 2.8.1 + optional: true '@aws-sdk/token-providers@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))': dependencies: @@ -14947,6 +14952,7 @@ snapshots: '@smithy/shared-ini-file-loader': 3.1.8 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@aws-sdk/types@3.662.0': dependencies: @@ -14959,10 +14965,12 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/util-endpoints': 2.1.3 tslib: 2.8.1 + optional: true '@aws-sdk/util-locate-window@3.568.0': dependencies: tslib: 2.8.1 + optional: true '@aws-sdk/util-user-agent-browser@3.662.0': dependencies: @@ -14970,6 +14978,7 @@ snapshots: '@smithy/types': 3.5.0 bowser: 2.11.0 tslib: 2.8.1 + optional: true '@aws-sdk/util-user-agent-node@3.662.0': dependencies: @@ -14977,6 +14986,7 @@ snapshots: '@smithy/node-config-provider': 3.1.8 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@babel/code-frame@7.24.7': dependencies: @@ -17317,7 +17327,7 @@ snapshots: '@kwsites/promise-deferred@1.1.1': {} - '@langchain/community@0.0.57(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(ws@8.18.0)': + '@langchain/community@0.0.57(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(ws@8.18.0)': dependencies: '@langchain/core': 0.1.63(openai@4.52.6) '@langchain/openai': 0.0.28 @@ -19703,26 +19713,11 @@ snapshots: dependencies: '@sinonjs/commons': 3.0.1 - '@sinonjs/fake-timers@11.2.2': - dependencies: - '@sinonjs/commons': 3.0.1 - - '@sinonjs/fake-timers@13.0.2': - dependencies: - '@sinonjs/commons': 3.0.1 - - '@sinonjs/samsam@8.0.2': - dependencies: - '@sinonjs/commons': 3.0.1 - lodash.get: 4.4.2 - type-detect: 4.1.0 - - '@sinonjs/text-encoding@0.7.3': {} - '@smithy/abort-controller@3.1.5': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/config-resolver@3.0.9': dependencies: @@ -19731,6 +19726,7 @@ snapshots: '@smithy/util-config-provider': 3.0.0 '@smithy/util-middleware': 3.0.7 tslib: 2.8.1 + optional: true '@smithy/core@2.4.7': dependencies: @@ -19744,6 +19740,7 @@ snapshots: '@smithy/util-middleware': 3.0.7 '@smithy/util-utf8': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/credential-provider-imds@3.2.4': dependencies: @@ -19752,6 +19749,7 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/url-parser': 3.0.7 tslib: 2.8.1 + optional: true '@smithy/eventstream-codec@3.1.6': dependencies: @@ -19759,29 +19757,41 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/util-hex-encoding': 3.0.0 tslib: 2.8.1 + optional: true + + '@smithy/eventstream-codec@4.0.1': + dependencies: + '@aws-crypto/crc32': 5.2.0 + '@smithy/types': 4.1.0 + '@smithy/util-hex-encoding': 4.0.0 + tslib: 2.8.1 '@smithy/eventstream-serde-browser@3.0.10': dependencies: '@smithy/eventstream-serde-universal': 3.0.9 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/eventstream-serde-config-resolver@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/eventstream-serde-node@3.0.9': dependencies: '@smithy/eventstream-serde-universal': 3.0.9 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/eventstream-serde-universal@3.0.9': dependencies: '@smithy/eventstream-codec': 3.1.6 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/fetch-http-handler@3.2.9': dependencies: @@ -19790,6 +19800,7 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/util-base64': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/hash-node@3.0.7': dependencies: @@ -19797,11 +19808,13 @@ snapshots: '@smithy/util-buffer-from': 3.0.0 '@smithy/util-utf8': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/invalid-dependency@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/is-array-buffer@2.2.0': dependencies: @@ -19810,12 +19823,18 @@ snapshots: '@smithy/is-array-buffer@3.0.0': dependencies: tslib: 2.8.1 + optional: true + + '@smithy/is-array-buffer@4.0.0': + dependencies: + tslib: 2.8.1 '@smithy/middleware-content-length@3.0.9': dependencies: '@smithy/protocol-http': 4.1.4 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/middleware-endpoint@3.1.4': dependencies: @@ -19826,6 +19845,7 @@ snapshots: '@smithy/url-parser': 3.0.7 '@smithy/util-middleware': 3.0.7 tslib: 2.8.1 + optional: true '@smithy/middleware-retry@3.0.22': dependencies: @@ -19838,16 +19858,19 @@ snapshots: '@smithy/util-retry': 3.0.7 tslib: 2.8.1 uuid: 9.0.1 + optional: true '@smithy/middleware-serde@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/middleware-stack@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/node-config-provider@3.1.8': dependencies: @@ -19855,6 +19878,7 @@ snapshots: '@smithy/shared-ini-file-loader': 3.1.8 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/node-http-handler@3.2.4': dependencies: @@ -19863,36 +19887,43 @@ snapshots: '@smithy/querystring-builder': 3.0.7 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/property-provider@3.1.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/protocol-http@4.1.4': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/querystring-builder@3.0.7': dependencies: '@smithy/types': 3.5.0 '@smithy/util-uri-escape': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/querystring-parser@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/service-error-classification@3.0.7': dependencies: '@smithy/types': 3.5.0 + optional: true '@smithy/shared-ini-file-loader@3.1.8': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/signature-v4@4.2.0': dependencies: @@ -19904,6 +19935,7 @@ snapshots: '@smithy/util-uri-escape': 3.0.0 '@smithy/util-utf8': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/smithy-client@3.3.6': dependencies: @@ -19913,30 +19945,39 @@ snapshots: '@smithy/types': 3.5.0 '@smithy/util-stream': 3.1.9 tslib: 2.8.1 + optional: true '@smithy/types@3.5.0': dependencies: - tslib: 2.6.2 + tslib: 2.8.1 + + '@smithy/types@4.1.0': + dependencies: + tslib: 2.8.1 '@smithy/url-parser@3.0.7': dependencies: '@smithy/querystring-parser': 3.0.7 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/util-base64@3.0.0': dependencies: '@smithy/util-buffer-from': 3.0.0 '@smithy/util-utf8': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/util-body-length-browser@3.0.0': dependencies: tslib: 2.8.1 + optional: true '@smithy/util-body-length-node@3.0.0': dependencies: tslib: 2.8.1 + optional: true '@smithy/util-buffer-from@2.2.0': dependencies: @@ -19947,10 +19988,17 @@ snapshots: dependencies: '@smithy/is-array-buffer': 3.0.0 tslib: 2.8.1 + optional: true + + '@smithy/util-buffer-from@4.0.0': + dependencies: + '@smithy/is-array-buffer': 4.0.0 + tslib: 2.8.1 '@smithy/util-config-provider@3.0.0': dependencies: tslib: 2.8.1 + optional: true '@smithy/util-defaults-mode-browser@3.0.22': dependencies: @@ -19959,6 +20007,7 @@ snapshots: '@smithy/types': 3.5.0 bowser: 2.11.0 tslib: 2.8.1 + optional: true '@smithy/util-defaults-mode-node@3.0.22': dependencies: @@ -19969,27 +20018,36 @@ snapshots: '@smithy/smithy-client': 3.3.6 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/util-endpoints@2.1.3': dependencies: '@smithy/node-config-provider': 3.1.8 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/util-hex-encoding@3.0.0': dependencies: tslib: 2.8.1 + optional: true + + '@smithy/util-hex-encoding@4.0.0': + dependencies: + tslib: 2.8.1 '@smithy/util-middleware@3.0.7': dependencies: '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/util-retry@3.0.7': dependencies: '@smithy/service-error-classification': 3.0.7 '@smithy/types': 3.5.0 tslib: 2.8.1 + optional: true '@smithy/util-stream@3.1.9': dependencies: @@ -20001,10 +20059,12 @@ snapshots: '@smithy/util-hex-encoding': 3.0.0 '@smithy/util-utf8': 3.0.0 tslib: 2.8.1 + optional: true '@smithy/util-uri-escape@3.0.0': dependencies: tslib: 2.8.1 + optional: true '@smithy/util-utf8@2.3.0': dependencies: @@ -20015,6 +20075,12 @@ snapshots: dependencies: '@smithy/util-buffer-from': 3.0.0 tslib: 2.8.1 + optional: true + + '@smithy/util-utf8@4.0.0': + dependencies: + '@smithy/util-buffer-from': 4.0.0 + tslib: 2.8.1 '@solid-primitives/trigger@1.1.0(solid-js@1.8.7)': dependencies: @@ -20454,12 +20520,6 @@ snapshots: '@types/shimmer@1.2.0': {} - '@types/sinon@17.0.3': - dependencies: - '@types/sinonjs__fake-timers': 8.1.5 - - '@types/sinonjs__fake-timers@8.1.5': {} - '@types/stack-utils@2.0.3': {} '@types/statuses@2.0.5': {} @@ -21547,11 +21607,7 @@ snapshots: '@fastify/error': 4.0.0 fastq: 1.17.1 - aws-sdk-client-mock@4.0.2: - dependencies: - '@types/sinon': 17.0.3 - sinon: 18.0.1 - tslib: 2.6.2 + aws4fetch@1.0.20: {} axe-core@4.10.0: {} @@ -21715,7 +21771,8 @@ snapshots: boolbase@1.0.0: {} - bowser@2.11.0: {} + bowser@2.11.0: + optional: true boxen@7.1.1: dependencies: @@ -22518,8 +22575,6 @@ snapshots: diff@4.0.2: {} - diff@5.2.0: {} - diff@7.0.0: {} digest-fetch@1.3.0: @@ -23505,6 +23560,7 @@ snapshots: fast-xml-parser@4.4.1: dependencies: strnum: 1.0.5 + optional: true fastify@5.1.0: dependencies: @@ -25112,8 +25168,6 @@ snapshots: object.assign: 4.1.5 object.values: 1.1.7 - just-extend@6.2.0: {} - jwa@2.0.0: dependencies: buffer-equal-constant-time: 1.0.1 @@ -25195,10 +25249,10 @@ snapshots: kolorist@1.8.0: {} - langchain@0.1.36(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(fast-xml-parser@4.4.1)(ignore@5.3.2)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(playwright@1.46.0)(ws@8.18.0): + langchain@0.1.36(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(fast-xml-parser@4.4.1)(ignore@5.3.2)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(playwright@1.46.0)(ws@8.18.0): dependencies: '@anthropic-ai/sdk': 0.9.1 - '@langchain/community': 0.0.57(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(ws@8.18.0) + '@langchain/community': 0.0.57(@aws-crypto/sha256-js@5.2.0)(@aws-sdk/client-bedrock-runtime@3.663.0)(@aws-sdk/credential-provider-node@3.662.0(@aws-sdk/client-sso-oidc@3.662.0(@aws-sdk/client-sts@3.662.0))(@aws-sdk/client-sts@3.662.0))(@smithy/util-utf8@2.3.0)(@upstash/redis@1.34.3)(@vercel/kv@0.2.4)(ioredis@5.4.1)(jsdom@24.0.0)(lodash@4.17.21)(openai@4.52.6)(ws@8.18.0) '@langchain/core': 0.1.63(openai@4.52.6) '@langchain/openai': 0.0.28 '@langchain/textsplitters': 0.0.2(openai@4.52.6) @@ -25438,8 +25492,6 @@ snapshots: lodash.defaults@4.2.0: {} - lodash.get@4.4.2: {} - lodash.isarguments@3.1.0: {} lodash.isequal@4.5.0: {} @@ -26209,14 +26261,6 @@ snapshots: - '@babel/core' - babel-plugin-macros - nise@6.1.1: - dependencies: - '@sinonjs/commons': 3.0.1 - '@sinonjs/fake-timers': 13.0.2 - '@sinonjs/text-encoding': 0.7.3 - just-extend: 6.2.0 - path-to-regexp: 8.2.0 - nitropack@2.10.4(@upstash/redis@1.34.3)(typescript@5.6.3): dependencies: '@cloudflare/kv-asset-handler': 0.3.4 @@ -28006,15 +28050,6 @@ snapshots: dependencies: is-arrayish: 0.3.2 - sinon@18.0.1: - dependencies: - '@sinonjs/commons': 3.0.1 - '@sinonjs/fake-timers': 11.2.2 - '@sinonjs/samsam': 8.0.2 - diff: 5.2.0 - nise: 6.1.1 - supports-color: 7.2.0 - sirv@2.0.4: dependencies: '@polka/url': 1.0.0-next.25 @@ -28278,7 +28313,8 @@ snapshots: dependencies: js-tokens: 9.0.1 - strnum@1.0.5: {} + strnum@1.0.5: + optional: true strtok3@6.3.0: dependencies: @@ -29110,8 +29146,6 @@ snapshots: type-detect@4.0.8: {} - type-detect@4.1.0: {} - type-fest@0.20.2: {} type-fest@0.21.3: {} diff --git a/turbo.json b/turbo.json index 7a3c0d58e7bb..12fcdc032923 100644 --- a/turbo.json +++ b/turbo.json @@ -7,8 +7,9 @@ "env": [ "ANTHROPIC_API_KEY", "ASSISTANT_ID", - "AWS_REGION", "AWS_ACCESS_KEY_ID", + "AWS_ACCOUNT_ID", + "AWS_REGION", "AWS_SECRET_ACCESS_KEY", "BASETEN_API_KEY", "CEREBRAS_API_KEY",