Merge pull request #2505 from philcallister/philcallister-feature-api…

…gw-websocket-api-bedrock-streaming-rust-cdk

New serverless pattern - apigw-websocket-api-bedrock-streaming-rust-cdk

apigw-websocket-api-bedrock-streaming-rust-cdk/.gitignore

@@ -0,0 +1,10 @@
+*.js
+!jest.config.js
+*.d.ts
+node_modules
+.DS_Store
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+

apigw-websocket-api-bedrock-streaming-rust-cdk/README.md

@@ -0,0 +1,109 @@
+# Streaming Amazon Bedrock response with Amazon API Gateway WebSocket API and AWS Lambda
+
+This CDK application demonstrates a simple, serverless approach to integrating Amazon Bedrock with AWS Lambda and Amazon API Gateway. Written in Rust, it showcases how to efficiently stream responses from Amazon Bedrock to a client via WebSocket connections. The example serves as a practical illustration of implementing real-time, serverless communication between Bedrock's GenAI capabilities and a client application.
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [Node and NPM](https://nodejs.org/en/download/) installed
+* [AWS Cloud Development Kit](https://docs.aws.amazon.com/cdk/latest/guide/cli.html) (AWS CDK) installed
+* [Docker](https://docs.docker.com/engine/install/) installed and running locally (needed for Rust cross-platform Lambda build)
+* [Rust](https://www.rust-lang.org/) 🦀 installed with v1.79.0 or higher
+* [Cargo Lambda](https://www.cargo-lambda.info/) installed
+* [cross](https://github.com/cross-rs/cross) compilation installed for Cargo Lambda: `cargo install cross --git https://github.com/cross-rs/cross`
+* [wscat](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-wscat.html) installed for CLI WebSocket capabilities
+
+
+## Amazon Bedrock Setup Instructions
+
+You must request access to the Bedrock LLM model before you can use it. This example uses `Claude 3 Sonnet`, so make sure you have `Access granted` to this model. For more information, see [Model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html).
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ```bash
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern's CDK directory:
+    ```bash
+    cd apigw-websocket-api-bedrock-streaming-rust-cdk/cdk
+    ```
+3. From the command line, use npm to install the development dependencies:  
+    ```bash
+    npm install
+    ```
+4. If you haven't done so previously for this account, run this command to bootstrap CDK:
+    ```bash
+    cdk bootstrap
+    ```
+5.  Review the CloudFormation template that CDK generates for your stack using the following AWS CDK CLI command:
+    ```bash
+    cdk synth
+    ```
+6. Use AWS CDK to deploy your AWS resources
+    ```bash
+    cdk deploy
+    ```
+
+    After the deployment completes, note the URL in the `Outputs` section at the end. The `BedrockStreamerStack.WebSocketURL` followed by the WebSocket URL will be used to connect to API Gateway. It should look something like `wss://{YOUR_API_ID_HERE}.execute-api.{YOUR_REGION_HERE}.amazonaws.com/prod`
+
+## How it works
+
+This pattern establishes a WebSocket connection to Amazon API Gateway. When requests are made to this API, API Gateway routes them to an AWS Lambda function. The Lambda function then initiates a streaming request to Amazon Bedrock using the [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html) API. This allows the response from the LLM in Bedrock to start streaming back to the Lambda function as soon as generation begins, without waiting for the entire response to be ready.
+
+Generating Bedrock responses often takes a long time. By using a streaming approach, the Lambda function can immediately start processing the incoming response and writing the data chunks to the API Gateway WebSocket. The WebSocket then delivers these chunks in real-time to the connected client, providing an extremely interactive user experience.
+
+## Testing
+
+1. From your terminal, use `wscat` to connect to API Gateway using the WebSocket API and generate a short story about `CATS` by entering the `{"storyType": "CATS"}` line after `wscat` startup.
+    ```bash
+    # Connect to the API Gateway via WebSocket
+    wscat -c <API_GATEWAY_URL_FROM_CDK_OUTPUT>
+
+    Connected (press CTRL+C to quit)
+    > {"storyType": "CATS"} <--- ENTER THIS...PRESS RETURN
+    < {"type":"other","message":null}
+    < {"type":"text","message":"Here"}
+    < {"type":"text","message":" is"}
+    < {"type":"text","message":" a"}
+    < {"type":"text","message":" very"}
+    < {"type":"text","message":" short"}
+    < {"type":"text","message":" story"}
+    < {"type":"text","message":" about"}
+    < {"type":"text","message":" cats"}
+    < {"type":"text","message":":"}
+    < {"type":"text","message":"\n\nMitt"}
+    < {"type":"text","message":"ens"}
+    < {"type":"text","message":" cur"}
+    < {"type":"text","message":"le"}
+    < {"type":"text","message":"d up"}
+    < {"type":"text","message":" on"}
+    < {"type":"text","message":" the"}
+    < {"type":"text","message":" window"}
+    < {"type":"text","message":"s"}
+    < {"type":"text","message":"ill"}
+    < {"type":"text","message":","}
+    .
+    .
+    < {"type":"other","message":null}
+    < {"type":"other","message":null}
+    < {"type":"other","message":null}
+    > ⏎ <--- CTRL+C HERE>
+    ```
+2. As the `wscat` CLI says, press `CTRL+C` to disconnect
+
+## Cleanup
+
+You can use the following commands to destroy the AWS resources created during deployment. This assumes you're currently at the `apigw-websocket-api-bedrock-streaming-rust-cdk/cdk` directory in your terminal:
+
+```bash
+cdk destroy
+```
+----
+Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

apigw-websocket-api-bedrock-streaming-rust-cdk/apigw-websocket-api-bedrock-streaming-rust-cdk.json

@@ -0,0 +1,93 @@
+{
+    "title": "Streaming Amazon Bedrock response with Amazon API Gateway",
+    "description": "Stream an Amazon Bedrock LLM response with API Gateway WebSocket API and AWS Lambda function .",
+    "language": "Rust",
+    "level": "200",
+    "framework": "CDK",
+    "introBox": {
+        "headline": "How it works",
+        "text": [
+            "This pattern establishes a WebSocket connection to Amazon API Gateway. When requests are made to this API, API Gateway routes them to an AWS Lambda function.",
+            "The Lambda function then initiates a streaming request to Amazon Bedrock using the ConverseStream API. This allows the response from the LLM in Bedrock to start streaming back to the Lambda function as soon as generation begins, without waiting for the entire response to be ready."
+        ]
+    },
+    "gitHub": {
+        "template": {
+            "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-websocket-api-bedrock-streaming-rust-cdk",
+            "templateURL": "serverless-patterns/apigw-websocket-api-bedrock-streaming-rust-cdk",
+            "projectFolder": "apigw-websocket-api-bedrock-streaming-rust-cdk",
+            "templateFile": "cdk/lib/bedrock-streamer-stack.ts"
+        }
+    },
+    "resources": {
+        "bullets": [
+            {
+                "text": "Working with WebSocket APIs",
+                "link": "https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api.html"
+            },
+            {
+                "text": "Amazon Bedrock",
+                "link": "https://aws.amazon.com/bedrock/"
+            },
+            {
+                "text": "Cloud Development Kit",
+                "link": "https://docs.aws.amazon.com/cdk/v2/guide/home.html"
+            },
+            {
+                "text": "AWS SDK for Rust",
+                "link": "https://aws.amazon.com/sdk-for-rust/"
+            }
+        ]
+    },
+    "deploy": {
+        "text": [
+            "cdk deploy"
+        ]
+    },
+    "testing": {
+        "text": [
+            "See the GitHub repo for detailed testing instructions."
+        ]
+    },
+    "cleanup": {
+        "text": [
+            "Delete the stack: <code>cdk destroy</code>."
+        ]
+    },
+    "authors": [
+        {
+            "name": "Phil Callister",
+            "image": "https://media.licdn.com/dms/image/v2/D5603AQHElNV-QFIGQw/profile-displayphoto-shrink_200_200/profile-displayphoto-shrink_200_200/0/1725218566874?e=1735171200&v=beta&t=gr82L1aMjvNX8CZn1Nitg3fViTUYoPK_xnXNc8CdohQ",
+            "bio": "I'm an Enterprise Solutions Architect at AWS, with a focus on Financial Services. As a passionate builder, I enjoy helping customers create innovative solutions to achieve their business objectives.",
+            "linkedin": "philcallister"
+        }
+    ],
+    "patternArch": {
+        "icon1": {
+            "x": 20,
+            "y": 50,
+            "service": "apigw",
+            "label": "API Gateway Websocket API"
+        },
+        "icon2": {
+            "x": 50,
+            "y": 50,
+            "service": "lambda",
+            "label": "AWS Lambda"
+        },
+        "icon3": {
+            "x": 80,
+            "y": 50,
+            "service": "bedrock",
+            "label": "Amazon Bedrock"
+        },
+        "line1": {
+            "from": "icon1",
+            "to": "icon2"
+        },
+        "line3": {
+            "from": "icon2",
+            "to": "icon3"
+        }
+    }
+}

apigw-websocket-api-bedrock-streaming-rust-cdk/cdk/.npmignore

@@ -0,0 +1,6 @@
+*.ts
+!*.d.ts
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out

apigw-websocket-api-bedrock-streaming-rust-cdk/cdk/bin/bedrock-streamer.ts

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+import 'source-map-support/register';
+import * as cdk from 'aws-cdk-lib';
+import { BedrockStreamerStack } from '../lib/bedrock-streamer-stack';
+
+const app = new cdk.App();
+const description = "Serverlessland Bedrock streamining pattern. (uksb-1tthgi812) (tag:apigw-websocket-api-bedrock-streamining-rust-cdk)"
+new BedrockStreamerStack(app, 'BedrockStreamerStack', {description:description
+  /* If you don't specify 'env', this stack will be environment-agnostic.
+   * Account/Region-dependent features and context lookups will not work,
+   * but a single synthesized template can be deployed anywhere. */
+
+  /* Uncomment the next line to specialize this stack for the AWS Account
+   * and Region that are implied by the current CLI configuration. */
+  // env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
+
+  /* Uncomment the next line if you know exactly what Account and Region you
+   * want to deploy the stack to. */
+  // env: { account: '123456789012', region: 'us-east-1' },
+
+  /* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
+});

apigw-websocket-api-bedrock-streaming-rust-cdk/cdk/cdk.json

@@ -0,0 +1,71 @@
+{
+  "app": "npx ts-node --prefer-ts-exts bin/bedrock-streamer.ts",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "**/*.d.ts",
+      "**/*.js",
+      "tsconfig.json",
+      "package*.json",
+      "yarn.lock",
+      "node_modules",
+      "test"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
+    "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
+    "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
+    "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
+    "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
+    "@aws-cdk/aws-eks:nodegroupNameAttribute": true,
+    "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
+    "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
+    "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false
+  }
+}

apigw-websocket-api-bedrock-streaming-rust-cdk/cdk/lib/bedrock-streamer-lambda/.gitignore

@@ -0,0 +1 @@
+/target

apigw-websocket-api-bedrock-streaming-rust-cdk/cdk/lib/bedrock-streamer-lambda/Cargo.toml

@@ -0,0 +1,24 @@
+[package]
+name = "bedrock-streamer"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
+aws_lambda_events = { version = "0.11.1", default-features = false, features = ["apigw"] }
+aws-config = "1.1.1"
+aws-sdk-bedrockruntime = "1.53.0"
+aws-sdk-apigatewaymanagement = "1.39.0"
+bytes = "1.5.0"
+http = "1.1.0"
+lambda_runtime = "0.9.1"
+serde = { version = "1.0.193", features = ["derive"] }
+serde_json = "1.0.108"
+tokio = { version = "1.34.0", features = ["full"] }
+tracing = { version = "0.1", features = ["log"] }
+tracing-subscriber = { version = "0.3", default-features = false, features = ["fmt"] }
+
+[profile.release]
+opt-level = "z"
+strip = true
+lto = true
+codegen-units = 1