Skip to content

Latest commit

 

History

History
57 lines (34 loc) · 1.24 KB

AWS.md

File metadata and controls

57 lines (34 loc) · 1.24 KB
Raw

Running on Amazon WebServices

Creating a kafka-cluster with monitoring tools setup in AWS. No support of SSL or ACLs.

⚠️ Only differences to local Vagrant version are described.

Init

Adapt ansible/inventories/aws/hosts according to your setup.

Setup

cd ansible

ansible-playbook -i inventories/aws/ cluster.yml

Connections

Name
Kafka Brokers kafka-1:9092,kafka-2:9092,kafka-3:9092

Kafka

Either ssh into one of your machines where Kafka is installed or install Kafka CLI tools locally.

Topic Creation

kafka-topics.sh --zookeeper kafka-1:2181 --create --replication-factor 1 --partitions 4 --topic sample

Producer

kafka-console-producer.sh --broker-list kafka-1:9092,kafka-3:9092 --topic sample

Hey, is Kafka up and running?

Consumer

kafka-console-consumer.sh --bootstrap-server kafka-1:9092,kafka-3:9092 --topic sample --from-beginning

Hey, is Kafka up and running?

Producer Perf Test

kafka-producer-perf-test.sh --producer-props bootstrap.servers="kafka-1:9092,kafka-2:9092,kafka-3:9092" --topic sample --num-records 2000 --throughput 100 --record-size 256