Merge pull request #1716 from nihilg/nihilg-feature-aurora-serverless…

…-global-db-cdk

New serverless pattern - Amazon Aurora Serverless V2 Primary to Secondary Global Database

aurora-serverless-global-db-cdk/.gitignore

@@ -0,0 +1,9 @@
+!jest.config.js
+*.d.ts
+node_modules
+.DS_Store
+package-lock.json
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out

aurora-serverless-global-db-cdk/.npmignore

@@ -0,0 +1,6 @@
+*.ts
+!*.d.ts
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out

aurora-serverless-global-db-cdk/README.md

@@ -0,0 +1,115 @@
+# Amazon Aurora Serverless V2 Primary to Secondary Global Database
+The pattern creates a serverless global database cluster enabling data replication from the primary to secondary cluster. The regional primary cluster contains a serverless db instance supporting both writes and reads. The regional secondary cluster contains a serverless db instance supporting only reads. In the unlikely event of a regional degradation or outage, the secondary region can be promoted to read and write capabilities in less than 1 minute. Also the pattern adopts the multiple stack capability of CDK to provision the resources across the primary and secondary regions. You can deploy each stack indvidually or deploy all the stacks using --all option.
+
+Learn more about this pattern at Serverless Land Patterns: << Add the live URL here >>
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [Node and NPM](https://nodejs.org/en/download/) installed.
+* [AWS CDK](https://docs.aws.amazon.com/cdk/latest/guide/cli.html) installed and configured
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern directory:
+    ```
+    cd aurora-serverless-global-db-cdk
+    ```
+3. Install dependencies:
+    ```
+    npm install
+    ```
+4. Configure AWS CDK to bootstrap the AWS account, primary region and secondary region :
+    ```
+    cdk bootstrap 111111111111/eu-west-1
+    cdk bootstrap 111111111111/eu-west-2
+    ```
+5. From the command line, use AWS CDK to deploy the all the stacks synthesized: 
+    ```
+    cdk deploy --all
+    ```
+    Alternatively you can also deploy each stack individually. From the command line, use AWS CDK to deploy each stack: 
+    ```
+    cdk deploy aurora-global-cluster
+    cdk deploy primary-cluster
+    cdk deploy secondary-cluster
+    cdk deploy primary-test-app
+    cdk deploy secondary-test-app
+    ```
+7. Note the outputs from the CDK deployment process. These contain the primary and secondary URLs which are used for testing.
+
+## How it works
+
+- The template in this pattern synthesizes five stacks for deployment using the multiple stack approach of CDK. 
+- The first stack named aurora-global-cluster creates the Aurora Global Cluster. 
+- The second stack named primary-cluster deploys the primary cluster with a serverless v2 instance in the primary region defined.
+- The third stack named secondary-cluster deploys the secondary cluster with a serverless v2 instance in the secondary region defined.
+- The fourth and fifth stack named primary-test-app and secondary-test-app deploys a fargate container with a nodejs app for testing the global database tables. 
+- The primary cluster supports both write and read operations. The secondary cluster supports read operation only.
+- Once you deploy all the stacks, you have built a global database that automatically replicates data from the primary to the secondary region.
+
+## Testing
+
+Note down the primary-test-app.FargateServiceURL and secondary-test-app.FargateServiceURL values from the CDK output and update them in each of the test commands below. 
+
+1.  Initialize the Global Database by creating a table 
+
+    ```
+    curl primary-test-app.FargateServiceURL/init
+    ```
+
+    Expected Response :
+    "Task table created.."
+
+2. Write a task into the Primary Cluster 
+    ```
+    curl -X POST primary-test-app.FargateServiceURL/tasks -H 'Content-Type: application/json' -d '{"name":"Task1","status":"created"}'
+    ```
+    Expected Response : Task added with ID: 1
+3. Read the tasks from the Secondary cluster 
+    ```
+    curl secondary-test-app.FargateServiceURL/tasks 
+    ```
+    Expected Response : 
+    [{"id":1,"name":"Task1","status":"created"}]
+
+4. Attempt to write a task into the Secondary cluster 
+    ```
+    curl -X POST secondary-test-app.FargateServiceURL/tasks -H 'Content-Type: application/json' -d '{"name":"Task1","status":"created"}'
+    ```
+    Expected Response : error : cannot execute INSERT in a read-only transaction
+    
+    Note : You can enable Write forwarding feature to continue to use the Secondary cluster endpoint for write transactions as well. 
+
+5. You can also try to update and delete the task on the Primary cluster 
+    ```
+    curl -X PUT primary-test-app.FargateServiceURL/tasks/1 -H 'Content-Type: application/json' -d '{"name":"Task1","status":"in-progress"}'
+
+    curl -X DELETE primary-test-app.FargateServiceURL/tasks/1
+
+    ```
+6. Check if the task is deleted from the Secondary cluster 
+    ```
+    curl secondary-test-app.FargateServiceURL/tasks 
+    ```
+    Expected Response : 
+    []
+
+## Cleanup
+ 
+1. Delete the stack
+    ```
+    cdk destroy --all 
+    ```
+----
+Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

aurora-serverless-global-db-cdk/bin/aurora-global-db-multistack.ts

@@ -0,0 +1,50 @@
+import { App } from 'aws-cdk-lib';
+import { AuroraGlobalClusterStack } from '../lib/aurora-global-cluster-stack';
+import { AuroraRegionalClusterStack } from '../lib/aurora-regional-cluster-stack';
+import { FargateTestAppStack } from '../lib/fargate-test-app-stack';
+
+const app = new App();
+
+const account = app.node.tryGetContext('account') || process.env.CDK_INTEG_ACCOUNT || process.env.CDK_DEFAULT_ACCOUNT;
+const primaryRegion = { account: account, region: 'eu-west-1' };
+const secondaryRegion = { account: account, region: 'eu-west-2' };
+
+const globalCluster = new AuroraGlobalClusterStack(app, "aurora-global-cluster", {
+    env: primaryRegion
+});
+
+const primaryclusterstack = new AuroraRegionalClusterStack(app, `primary-cluster`, {
+    env: primaryRegion, cfnGlobalCluster: globalCluster.cfnGlobalCluster, isPrimary: true
+});
+
+const secondaryclusterstack = new AuroraRegionalClusterStack(app, `secondary-cluster`, {
+    env: secondaryRegion, cfnGlobalCluster: globalCluster.cfnGlobalCluster, isPrimary: false
+});
+
+primaryclusterstack.addDependency(globalCluster);
+secondaryclusterstack.addDependency(primaryclusterstack)
+
+const primarytestappstack = new FargateTestAppStack(app, `primary-test-app`, {
+    env: primaryRegion,
+    endpoint: primaryclusterstack.endpoint,
+    port: primaryclusterstack.port,
+    vpc: primaryclusterstack.vpc,
+    isPrimary: true,
+    region: primaryclusterstack.region,
+    dbSecurityGroupId: primaryclusterstack.dbSecurityGroupId
+});
+
+const secondarytestappstack = new FargateTestAppStack(app, `secondary-test-app`, {
+    env: secondaryRegion,
+    endpoint: secondaryclusterstack.endpoint,
+    port: secondaryclusterstack.port,
+    vpc: secondaryclusterstack.vpc,
+    isPrimary: false,
+    region: primaryclusterstack.region,
+    dbSecurityGroupId: secondaryclusterstack.dbSecurityGroupId
+});
+
+primarytestappstack.addDependency(primaryclusterstack);
+secondarytestappstack.addDependency(secondaryclusterstack);
+
+app.synth();

aurora-serverless-global-db-cdk/cdk.json

@@ -0,0 +1,57 @@
+{
+  "app": "npx ts-node --prefer-ts-exts bin/aurora-global-db-multistack.ts",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "**/*.d.ts",
+      "**/*.js",
+      "tsconfig.json",
+      "package*.json",
+      "yarn.lock",
+      "node_modules",
+      "test"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-iam:standardizedServicePrincipals": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true
+  }
+}

aurora-serverless-global-db-cdk/example-pattern.json

@@ -0,0 +1,57 @@
+{
+  "title": "Amazon Aurora Serverless Primary to Secondary Global Database",
+  "description": "Create an Aurora Serverless Global database to replicate across regions",
+  "language": "TypeScript",
+  "level": "300",
+  "framework": "CDK",
+  "introBox": {
+    "headline": "How it works",
+    "text": [
+      "The pattern provisions an Amazon Aurora serverless v2 global database enabling data replication from the primary to secondary region. The regional primary cluster contains a serverless db instance and supports both writes and reads. The regional secondary cluster contains a serverless db instance and supports only reads. In the unlikely event of a regional degradation or outage, the secondary region can be promoted to read and write capabilities in less than 1 minute",
+      "Also the pattern adopts the multiple stack capability of CDK to provision the resources across the primary and secondary regions."
+    ]
+  },
+  "gitHub": {
+    "template": {
+      "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/aurora-serverless-global-db-cdk",
+      "templateURL": "serverless-patterns/aurora-serverless-global-db-cdk",
+      "projectFolder": "aurora-serverless-global-db-cdk",
+      "templateFile": "aurora-serverless-global-db-cdk/lib/aurora-regional-cluster-stack.ts"
+    }
+  },
+  "resources": {
+    "bullets": [
+      {
+        "text": "Using Aurora Serverless V2",
+        "link": "https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.html"
+      },
+      {
+        "text": "Amazon Aurora Global Database",
+        "link": "https://aws.amazon.com/rds/aurora/global-database/"
+      }
+    ]
+  },
+  "deploy": {
+    "text": [
+      "cdk deploy --all"
+    ]
+  },
+  "testing": {
+    "text": [
+      "See the GitHub repo for detailed testing instructions."
+    ]
+  },
+  "cleanup": {
+    "text": [
+      "Delete the stack: <code>cdk destroy --all</code>."
+    ]
+  },
+  "authors": [
+    {
+      "name": "Nihilson Gnanadason",
+      "image": "https://drive.google.com/file/d/1m82LvtdoipI9nI_q2G6qoRGpZ36JNzHU/view?usp=sharing",
+      "bio": "Nihilson is a Sr.Solutions Architect at AWS working with ISVs in the UK to build, run, and scale their software products on AWS.",
+      "linkedin": "https://www.linkedin.com/in/nihilson/"
+    }
+  ]
+}

aurora-serverless-global-db-cdk/fargate/Dockerfile

@@ -0,0 +1,9 @@
+FROM --platform=linux/amd64 node:16-alpine
+WORKDIR /usr
+COPY package.json ./
+COPY src ./src
+RUN npm install
+RUN npm install pg
+COPY . .
+EXPOSE 80
+CMD ["npm","start"]

aurora-serverless-global-db-cdk/fargate/package.json

@@ -0,0 +1,17 @@
+{
+  "name": "fargate",
+  "version": "1.0.0",
+  "description": "Fargate ECS container to run an express app",
+  "main": "src/index.js",
+  "scripts": {
+    "build": "tsc",
+    "start": "node src/index.js"
+  },
+  "author": "nihilson",
+  "license": "ISC",
+  "dependencies": {
+    "express": "^4.18.2",
+    "pg": "^8.11.3",
+    "aws-sdk": "latest"
+  }
+}