Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create HA Amazon MQ broker #412

Merged
merged 3 commits into from
Jan 31, 2025
Merged

create HA Amazon MQ broker #412

merged 3 commits into from
Jan 31, 2025

Conversation

DaMandal0rian
Copy link
Member

@DaMandal0rian DaMandal0rian commented Jan 30, 2025

User description

Managed RabbitMQ with Amazon MQ with high-availability, replication and optimized EBS volume


PR Type

Enhancement


Description

  • Added high-availability RabbitMQ brokers using Amazon MQ with multi-AZ deployment.

  • Configured security groups, KMS encryption, and IAM roles for RabbitMQ.

  • Introduced new variables and outputs for RabbitMQ configuration and access.

  • Enabled logging and replication for RabbitMQ brokers.


Changes walkthrough 📝

Relevant files
Enhancement
broker.tf
Added RabbitMQ broker resources and configurations             

auto-drive/broker.tf

  • Added resources for primary and secondary RabbitMQ brokers.
  • Configured security groups for RabbitMQ brokers.
  • Added KMS key and alias for encryption.
  • Defined IAM roles and policies for CloudWatch logging.
  • +223/-0 
    outputs.tf
    Added RabbitMQ broker-related outputs                                       

    auto-drive/outputs.tf

  • Added outputs for RabbitMQ broker endpoints.
  • Included sensitive outputs for RabbitMQ credentials.
  • Added output for RabbitMQ instance type.
  • +34/-0   
    variables.tf
    Introduced RabbitMQ-related variables                                       

    auto-drive/variables.tf

  • Added variables for RabbitMQ usernames, instance type, and version.
  • Marked sensitive variables for RabbitMQ credentials.
  • +26/-0   

    Need help?
  • Type /help how to ... in the comments thread for any questions about PR-Agent usage.
  • Check out the documentation for more information.
  • Managed RabbitMQ with Amazon MQ with high-availability, replication and optimized EBS volume
    Copy link

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 No relevant tests
    🔒 Security concerns

    Sensitive information exposure:
    The rabbitmq_password and usernames are marked as sensitive but are still being output. This could lead to accidental exposure if access controls are not properly enforced. Additionally, ensure that the IAM role and policy for CloudWatch logging are scoped to the minimum permissions required to avoid privilege escalation risks.

    ⚡ Recommended focus areas for review

    Sensitive Data Exposure

    The random_password.rabbitmq_password.result is directly used in multiple places, including outputs and user configurations. Ensure that this sensitive data is securely handled and not inadvertently exposed.

    resource "random_password" "rabbitmq_password" {
      length           = 15
      special          = true # Includes special characters
      override_special = "!@#$%^&*()-_=+[]{}<>:?"
    }
    
    variable "private_subnet_cidrs" {
      default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    }
    
    resource "aws_mq_broker" "rabbitmq_broker_primary" {
      broker_name        = "auto-drive-rabbitmq-broker-primary"
      engine_type        = "RabbitMQ"
      engine_version     = var.rabbitmq_version
      host_instance_type = var.rabbitmq_instance_type # t3.micro is the smallest instance type available for Amazon MQ, use mq.m5.large for production
      security_groups    = [aws_security_group.rabbitmq_broker_primary.id]
      deployment_mode    = "ACTIVE_STANDBY_MULTI_AZ"
      storage_type       = "ebs"
      apply_immediately  = true
    
      subnet_ids          = module.vpc.private_subnets # Use private subnets from VPC module
      publicly_accessible = false
      encryption_options {
        use_aws_owned_key = false
        kms_key_id        = aws_kms_key.mq_kms_key.arn
      }
    
      logs {
        general = true
        audit   = false
      }
    
      maintenance_window_start_time {
        day_of_week = "Sunday"
        time_of_day = "03:00"
        time_zone   = "UTC"
      }
    
      user {
        username = var.rabbitmq_username
        password = random_password.rabbitmq_password.result
      }
    
      user {
        username         = var.rabbitmq_replication_username
        password         = random_password.rabbitmq_password.result
        replication_user = true
      }
    
      tags = {
        Environment = "Production"
        Application = "AutoDrive"
      }
    }
    
    resource "aws_mq_broker" "rabbitmq_broker_secondary" {
      broker_name        = "auto-drive-rabbitmq-broker-secondary"
      engine_type        = "RabbitMQ"
      engine_version     = var.rabbitmq_version
      host_instance_type = var.rabbitmq_instance_type # t3.micro is the smallest instance type available for Amazon MQ, use mq.m5.large for production
      security_groups    = [aws_security_group.rabbitmq_broker_secondary.id]
      deployment_mode    = "ACTIVE_STANDBY_MULTI_AZ"
      storage_type       = "ebs"
      apply_immediately  = true
    
      data_replication_mode               = "CRDR"
      data_replication_primary_broker_arn = aws_mq_broker.rabbitmq_broker_primary.arn
    
      subnet_ids          = module.vpc.private_subnets # Use private subnets from VPC module
      publicly_accessible = false
      encryption_options {
        use_aws_owned_key = false
        kms_key_id        = aws_kms_key.mq_kms_key.arn
      }
    
      logs {
        general = true
        audit   = false
      }
    
      maintenance_window_start_time {
        day_of_week = "Sunday"
        time_of_day = "04:00"
        time_zone   = "UTC"
      }
    
      user {
        username = var.rabbitmq_username
        password = random_password.rabbitmq_password.result
      }
    
      user {
        username         = var.rabbitmq_replication_username
        password         = random_password.rabbitmq_password.result
        replication_user = true
      }
    
      tags = {
        Environment = "Production"
        Application = "AutoDrive"
      }
    }
    
    # Security Group for RabbitMQ Primary Broker
    resource "aws_security_group" "rabbitmq_broker_primary" {
      name        = "rabbitmq-primary-sg"
      description = "Security group for RabbitMQ primary broker"
      vpc_id      = module.vpc.vpc_id
    
      ingress {
        from_port   = 5671
        to_port     = 5671
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      ingress {
        from_port   = 5672
        to_port     = 5672
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      egress {
        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      tags = {
        Name = "rabbitmq-primary-sg"
      }
    }
    
    # Security Group for RabbitMQ Secondary Broker
    resource "aws_security_group" "rabbitmq_broker_secondary" {
      name        = "rabbitmq-secondary-sg"
      description = "Security group for RabbitMQ secondary broker"
      vpc_id      = module.vpc.vpc_id
    
      ingress {
        from_port   = 5671
        to_port     = 5671
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      ingress {
        from_port   = 5672
        to_port     = 5672
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      egress {
        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      tags = {
        Name = "rabbitmq-secondary-sg"
      }
    }
    
    # KMS Key for Encryption
    resource "aws_kms_key" "mq_kms_key" {
      description             = "KMS key for RabbitMQ encryption"
      enable_key_rotation     = true
      deletion_window_in_days = 30
    }
    
    resource "aws_kms_alias" "mq_kms_alias" {
      name          = "alias/rabbitmq-broker-key"
      target_key_id = aws_kms_key.mq_kms_key.id
    }
    
    # IAM Role for CloudWatch Logging
    resource "aws_iam_role" "mq_cloudwatch_role" {
      name               = "mq-cloudwatch-logs-role"
      assume_role_policy = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "mq.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    EOF
    }
    
    resource "aws_iam_policy" "mq_cloudwatch_policy" {
      name        = "mq-cloudwatch-logs-policy"
      description = "Policy for allowing Amazon MQ to write logs to CloudWatch"
      policy      = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents"
          ],
          "Resource": "arn:aws:logs:*:*:*"
        }
      ]
    }
    EOF
    }
    
    resource "aws_iam_role_policy_attachment" "mq_cloudwatch_attach" {
      role       = aws_iam_role.mq_cloudwatch_role.name
      policy_arn = aws_iam_policy.mq_cloudwatch_policy.arn
    }
    Security Group Configuration

    The security groups for RabbitMQ brokers allow ingress traffic on ports 5671 and 5672 from all private subnets. Verify that this configuration aligns with the intended security posture and does not introduce vulnerabilities.

    # Security Group for RabbitMQ Primary Broker
    resource "aws_security_group" "rabbitmq_broker_primary" {
      name        = "rabbitmq-primary-sg"
      description = "Security group for RabbitMQ primary broker"
      vpc_id      = module.vpc.vpc_id
    
      ingress {
        from_port   = 5671
        to_port     = 5671
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      ingress {
        from_port   = 5672
        to_port     = 5672
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      egress {
        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      tags = {
        Name = "rabbitmq-primary-sg"
      }
    }
    
    # Security Group for RabbitMQ Secondary Broker
    resource "aws_security_group" "rabbitmq_broker_secondary" {
      name        = "rabbitmq-secondary-sg"
      description = "Security group for RabbitMQ secondary broker"
      vpc_id      = module.vpc.vpc_id
    
      ingress {
        from_port   = 5671
        to_port     = 5671
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      ingress {
        from_port   = 5672
        to_port     = 5672
        protocol    = "tcp"
        cidr_blocks = var.private_subnet_cidrs
      }
    
      egress {
        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
      }
    
      tags = {
        Name = "rabbitmq-secondary-sg"
      }
    Sensitive Outputs

    Sensitive information such as rabbitmq_password and usernames are being exposed as outputs. Confirm that these outputs are necessary and ensure proper access controls are in place.

    # RabbitMQ Broker Outputs
    output "rabbitmq_primary_endpoint" {
      description = "Primary RabbitMQ broker endpoint"
      value       = aws_mq_broker.rabbitmq_broker_primary.instances[0].endpoints[0]
    }
    
    output "rabbitmq_secondary_endpoint" {
      description = "Secondary RabbitMQ broker endpoint"
      value       = aws_mq_broker.rabbitmq_broker_secondary.instances[0].endpoints[0]
    }
    
    output "rabbitmq_instance_type" {
      description = "Instance type for RabbitMQ broker instances"
      value       = var.rabbitmq_instance_type
    }
    
    output "rabbitmq_username" {
      description = "RabbitMQ username"
      value       = var.rabbitmq_username
      sensitive   = true
    }
    
    output "rabbitmq_replication_username" {
      description = "RabbitMQ replication username"
      value       = var.rabbitmq_replication_username
      sensitive   = true
    }
    
    output "rabbitmq_password" {
      description = "RabbitMQ password"
      value       = random_password.rabbitmq_password.result
      sensitive   = true
    }

    Copy link

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Security
    Avoid reusing passwords for different users

    Ensure that the random_password.rabbitmq_password.result is not reused for both the
    primary user and the replication user to avoid potential security risks and
    conflicts.

    auto-drive/broker.tf [39-47]

     user {
       username = var.rabbitmq_username
       password = random_password.rabbitmq_password.result
     }
     
     user {
       username         = var.rabbitmq_replication_username
    -  password         = random_password.rabbitmq_password.result
    +  password         = random_password.rabbitmq_replication_password.result
       replication_user = true
     }
    Suggestion importance[1-10]: 9

    Why: Reusing the same password for both the primary user and the replication user poses a significant security risk. This suggestion addresses a critical issue by recommending the use of a separate password for the replication user, improving security and reducing potential conflicts.

    9
    Restrict outbound traffic in security groups

    Restrict the egress rule in the security groups to only allow necessary outbound
    traffic instead of permitting all traffic to enhance security.

    auto-drive/broker.tf [124-128]

     egress {
    -  from_port   = 0
    -  to_port     = 0
    -  protocol    = "-1"
    -  cidr_blocks = ["0.0.0.0/0"]
    +  from_port   = 5671
    +  to_port     = 5672
    +  protocol    = "tcp"
    +  cidr_blocks = var.private_subnet_cidrs
     }
    Suggestion importance[1-10]: 8

    Why: Restricting outbound traffic enhances security by limiting unnecessary exposure. This suggestion provides a clear and actionable improvement to the security group configuration.

    8
    Prevent exposure of sensitive outputs

    Avoid exposing sensitive outputs like rabbitmq_password in the outputs file, even if
    marked as sensitive, to reduce the risk of accidental exposure.

    auto-drive/outputs.tf [218-222]

    -output "rabbitmq_password" {
    -  description = "RabbitMQ password"
    -  value       = random_password.rabbitmq_password.result
    -  sensitive   = true
    -}
    +# Avoid exposing sensitive outputs like passwords
    Suggestion importance[1-10]: 7

    Why: Avoiding the exposure of sensitive outputs, even when marked as sensitive, reduces the risk of accidental leaks. This suggestion improves security, but it could have been more impactful if it proposed an alternative way to handle the sensitive data.

    7
    General
    Validate instance type for production readiness

    Add a condition to validate that the var.rabbitmq_instance_type is set to a
    production-ready instance type for non-development environments to prevent
    performance issues.

    auto-drive/broker.tf [15]

    -host_instance_type = var.rabbitmq_instance_type # t3.micro is the smallest instance type available for Amazon MQ, use mq.m5.large for production
    +host_instance_type = var.rabbitmq_instance_type
    +# Add validation logic to ensure production environments use mq.m5.large or higher
    Suggestion importance[1-10]: 6

    Why: Adding validation logic for the instance type ensures that production environments use a suitable instance type, preventing performance issues. However, the suggestion is not actionable as it does not provide specific implementation details.

    6

    clostao
    clostao previously approved these changes Jan 30, 2025
    @DaMandal0rian DaMandal0rian merged commit 6615dc0 into main Jan 31, 2025
    @DaMandal0rian DaMandal0rian deleted the add-rabbitmq branch January 31, 2025 08:16
    @DaMandal0rian
    Copy link
    Member Author

    closes #419

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants