Oracle database CDC (Change Data Capture)

Build clickstream analytics on AWS for your mobile and web applications

Terraform module to create AWS MSK (Managed Streaming for Kafka) resources 🇺🇦

This project provides and example of end to end data processing application created using the combination of Amazon Managed Streaming for Apache Kafka (Amazon MSK), AWS Fargate, AWS Lambda and Amazon DynamoDB. Business logic is implemented in Java and Typescript. The build and deployment of the application is fully automated using AWS CDK.

Terraform module which creates Msk Kafka Cluster on AWS

Create terraform module for deploying AWS MSK cluster

This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (MSK) and MSK Serverless into Apache Iceberg table in S3 with AWS Glue Streaming.

Terraform Kafka Provider

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK Serverless and MSK Connect (Debezium)

My AWS Playground

Pinterest's experiment analytics data pipeline which runs thousands of experiments per day and crunches billions of datapoints to provide valuable insights to improve the product.

Step by step guidance on how to setup MirrorMaker2 on AWS to perform data replication between 2 AWS MSK clusters

🌳 A sustainable Terraform Package which creates resources for Messaging Services (EventBridge, MSK, SNS, SQS) on AWS

A serverless offline plugin that enables AWS MSK events

AWS MSK Connect workshop helps first time users explore and get started to deploy fully managed Apache Kafka Connect workloads using Amazon MSK Connect.

Streaming data pipeline to continuously load data from an Amazon MSK or MSK Serverless cluster to Amazon S3 using Amazon Kinesis Data Firehose.

The CloudFormation Resource Provider Package For Amazon MSK Connect

Use-Case: Airline on-time performance

Terraform module to create kafka resource on AWS. AWS MSK (Managed Streaming for Apache Kafka) is a fully managed service that simplifies the deployment, management, and operation of Apache Kafka clusters. Apache Kafka is an open-source distributed streaming platform used for building real-time streaming data p

A Cloud based Reddit stock sentiment analyzer that analyzes overall sentiment from a configurable selection of stock subreddits for each stock. The architecture utilizes AWS MSK (Kafka), AWS EMR (PySpark) and AWS Lambda (Python 3) for maximum scalability and the OpenAI API for sentiment analysis through prompt engineering.