Realtime Election Voting System

This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to easily spin up the required services in Docker containers.

System Components

• main.py: This is the main Python script that creates the required tables on postgres (candidates, voters and votes), it also creates the Kafka topic and creates a copy of the votes table in the Kafka topic. It also contains the logic to consume the votes from the Kafka topic and produce data to voters_topic on Kafka.
• voting.py: This is the Python script that contains the logic to consume the votes from the Kafka topic (voters_topic), generate voting data and produce data to votes_topic on Kafka.

Prerequisites

• Python 3.9 or above installed on your machine
• Docker Compose installed on your machine
• Docker installed on your machine

Steps to Run

1. Clone this repository.
2. Navigate to the root containing the Docker Compose file.
3. Run the following command:

docker-compose up -d

This command will start Zookeeper, Kafka and Postgres containers in detached mode (-d flag). Kafka will be accessible at localhost:9092 and Postgres at localhost:5432.

Additional Configuration

If you need to modify Zookeeper configurations or change the exposed port, you can update the docker-compose.yml file according to your requirements.

Running the App

1. Install the required Python packages using the following command:

pip install -r requirements.txt

2. Creating the required tables on Postgres and generating voter information on Kafka topic:

python main.py

3. Consuming the voter information from Kafka topic, generating voting data and producing data to Kafka topic:

python voting.py