DORA Metrics and Pipeline Workflow Configuration Changes

Dora Metrics and Workflow Configuration Changes

Overview

Recent updates were made to our CI/CD pipeline to enhance support for Dora Metrics and optimize deployment workflows. Initially, the focus was on automating the build, test, and deployment process for the Go backend application within the golang boilerplate repository. The new setup now involves compressing Docker artifacts for each branch—dev, staging, and main—within the golang boilerplate repository and then decompressing and building the updated containers on the Telex app's remote server.

The old pipeline was triggered by pushes or pull requests to the dev branch and consists of two main jobs:

1. Build and Test Job:

   • Runs on ubuntu-latest.
   • Sets up PostgreSQL and Redis services for testing.
   • Builds the application and runs tests using the Go environment.
   • Ensures the application is properly started before running tests.

2. Deploy Job:

   • Executes only on pull requests, after the build and test job succeeds.
   • Uses SSH to deploy the application to a remote server.
   • Clones or pulls the latest code from the dev branch.
   • Runs a deployment script with environment-specific parameters.

Previous Development Workflow

In the previous workflow, the Docker image for the Go application was built using a Dockerfile. The pipeline was designed to handle deployments of several environment and involved the following steps:

1. Build and Test: The Go application was built and tested locally, using PostgreSQL and Redis as services within the CI pipeline.
2. Deployment: The application was deployed to a remote server by pulling the latest code from the dev branch and running a deployment script.

This setup was streamlined for multi environment, with the Dockerfile providing a uniform build process.

New Development Workflow

The new workflow leverages Docker Compose to manage environment-specific configurations. The new workflow is more robust and supports multiple environments by employing different Docker Compose files tailored for each environment. Here's an overview of the key changes:

1. Multi-Environment Support:

   • Separate Docker Compose files are used for Development, Staging, and Production, each configured with environment-specific settings.
   • Docker images are now built from the golang boilerplate server and pushed to the main telex app remote server, ensuring consistency across environments.

2. CI/CD Pipeline:

   • Build Docker Image: The pipeline builds the Docker image using the Docker Compose file for the specified environment. The image is then compressed and uploaded as an artifact.
   • Upload and Deploy: The image is transferred to the main telex app remote server, where it is decompressed and loaded. Docker Compose is then used to spin up the application in the appropriate environment.

Example of Workflow Changes

Old Development CI/CD Workflow:

The previous workflow included a single job to build, test, and deploy the Go application for the various environments. This setup utilized environment variables and services like PostgreSQL and Redis within the GitHub Actions workflow.

New Development CI/CD Workflow:

The new workflow is divided into multiple jobs:

1. Build Docker Image: Builds the Docker image for the Go application and saves it as a compressed file.
2. Upload Docker Image: Transfers the Docker image to the remote server.
3. Deploy Application: Runs the Docker Compose file on the remote server to deploy the application in various environments.

Docker Compose Example for Development

The following Docker Compose file is used to configure the Development environment:

name: golang_dev

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: development_user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: development_db
      POSTGRES_PORT: 5432
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 10s
      retries: 2

  redis:
    image: redis:latest

  backend:
    image: ${COMPOSE_PROJECT_NAME}
    build:
      context: .
    depends_on:
      - db
      - redis
    ports:
      - "8000:7000"
    env_file:
      - app.env

volumes:
  db_data:

This setup provides a standardized and scalable approach to managing deployments across different environments, ensuring that each environment is configured according to its specific requirements.