Skip to content

Latest commit

 

History

History

Exercise07

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
register-mysql.json
register-mysql.json
 
 

README.md

Debezium: Setup

Introduction

In this exercise we will setup a CDC architecture with the following components:

MySQL --> CDC (Debezium) --> Kafka

FYI, we will follow similar steps as defined in the Debezium tutorial, in case you want to have more details:

Pre-requisites

Run the docker containers with Docker Compose, starting the following containers:

docker compose up -d zookeeper kafka connect redpanda-console mysql adminer

Exercise

And install the Debezium connector in Kafka Connect:

# If you are using Cywin, Mac, Linux or a Windows version of "curl"
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:8083/connectors/ -d @Exercises/Exercise07/register-mysql.json

# If you are using PowerShell in Windows
Invoke-RestMethod -Uri http://localhost:8083/connectors/ -Method POST -Infile 'Exercises\Exercise07\register-mysql.json' -ContentType 'application/json'

This will make a call to the Kafka Connect REST API and deploy the Debezium connector defined in register-mysql.json.

Do some final checks:

  • Check that the connector has been created in the Kafka UI via the "Connectors" tab. You should see a connector named "inventory-connector" with status "RUNNING"
  • Check that you are able to access MySQL using the Adminer UI, and that the "inventory" database exists

NOTE: All the component URLs and credentials required are defined below in the "Understanding the components" section.

Understanding the components

In this installation we have the following components:

  • Kafka: Message broker we will use to send real-time update messages from Debezium
  • Zookeeper: Required for Kafka to run. You can ignore it for the time being
  • Kafka Connect: This is where Debezium will run and will connect to MySQL to get the real-time updates
  • Kafka UI (a.k.a. Redpanda Console): UI to visualize Kafka: topics, messages and connectors
  • MySQL: Database we will be using as the source
  • Adminer (MySQL UI): UI to view the database, tables, etc. No 100% required since you could do it using the command line
    • System: MySQL
    • Server: mysql
    • Username: mysqluser
    • Password: mysqlpw
    • Database: inventory

Resources