Skip to content

Latest commit

 

History

History
166 lines (131 loc) · 9.12 KB

README.md

File metadata and controls

166 lines (131 loc) · 9.12 KB
Raw

tf_aws_elasticsearch

Terraform module for deploying and managing Amazon Elasticsearch Service.

This module has two options for creating an Elasticsearch domain:

  1. Create an Elasticsearch domain with a public endpoint. Access policy is then based on the intersection of the following two criteria

    • source IP address
    • client IAM role

    See this Stack Overflow post for further discussion of access policies for Elasticsearch.

  2. Create an Elasticsearch domain and join it to a VPC. Access policy is then based on the intersection of the following two criteria:

    • security groups applied to Elasticsearch domain
    • client IAM role

If vpc_options option is set, Elasticsearch domain is created within a VPC. If not, Elasticsearch domain is created with a public endpoint

NOTE: You can either launch your domain within a VPC or use a public endpoint, but you can't do both. Considering this, adding or removing vpc_options will force DESTRUCTION of the old Elasticsearch domain and CREATION of a new one. More INFO - VPC support

Several options affect the resilience and scalability of your Elasticsearch domain. For a production deployment:

  • set instance_count to an even number (default: 6) greater than or equal to the dedicated_master_threshold (default: 10)
  • choose an instance_type that is not in the T2 family
  • set es_zone_awareness to true.

This will result in a cluster with three dedicated master nodes, balanced across two availability zones.

For a production deployment it may also make sense to use EBS volumes rather that instance storage; to do so, set ebs_volume_size greater than 0 and optionally specify a value for ebs_volume_type (right now the only supported values are gp2 and magnetic).

Terraform versions

Terraform 0.12. Pin module version to ~> v1.0. Submit pull-requests to master branch.

Terraform 0.11. Pin module version to ~> v0.0. Submit pull-requests to terraform011 branch.

Usage

Create Elasticsearch domain with public endpoint

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  management_public_ip_addresses = ["34.203.XXX.YYY"]
  instance_count                 = 16
  instance_type                  = "m4.2xlarge.elasticsearch"
  dedicated_master_type          = "m4.large.elasticsearch"
  es_zone_awareness              = true
  ebs_volume_size                = 100
}

Create Elasticsearch domain within a VPC and CloudWatch logs

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  vpc_options                    = {
    security_group_ids = ["sg-XXXXXXXX"]
    subnet_ids         = ["subnet-YYYYYYYY"]
  }
  instance_count                 = 1
  instance_type                  = "t2.medium.elasticsearch"
  dedicated_master_type          = "t2.medium.elasticsearch"
  es_zone_awareness              = false
  ebs_volume_size                = 35
  
  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"   # double quotes are required here
  }

  log_publishing_options = [
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "INDEX_SLOW_LOGS"
      enabled                  = true
    },
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "SEARCH_SLOW_LOGS"
      enabled                  = true
    },
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "ES_APPLICATION_LOGS"
      enabled                  = true
    }
  ]
}

Create small (4-node) Elasticsearch domain in a VPC with dedicated master nodes

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  vpc_options                    = {
    security_group_ids = ["sg-XXXXXXXX"]
    subnet_ids         = ["subnet-YYYYYYYY"]
  }
  instance_count                 = 4
  instance_type                  = "m4.2xlarge.elasticsearch"
  dedicated_master_threshold     = 4
  dedicated_master_type          = "m4.large.elasticsearch"
  es_zone_awareness              = true
  ebs_volume_size                = 100
}

Inputs

Name Description Type Default Required
advanced_options Map of key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your Elasticsearch domain on every apply. map(string) {} no
create_iam_service_linked_role Whether to create IAM service linked role for AWS ElasticSearch service. Can be only one per AWS account. bool "true" no
dedicated_master_threshold The number of instances above which dedicated master nodes will be used. Default: 10 number "10" no
dedicated_master_type ES instance type to be used for dedicated masters (default same as instance_type) string "false" no
domain_name Domain name for Elasticsearch cluster string "es-domain" no
domain_prefix String to be prefixed to search domain. Default: tf- string "tf-" no
ebs_volume_size Optionally use EBS volumes for data storage by specifying volume size in GB (default 0) number "0" no
ebs_volume_type Storage type of EBS volumes, if used (default gp2) string "gp2" no
encrypt_at_rest Enable encrption at rest (only specific instance family types support it: m4, c4, r4, i2, i3 default: false) bool "false" no
enforce_https Whether or not to require HTTPS. bool "false" no
es_version Version of Elasticsearch to deploy (default 5.1) string "5.1" no
es_zone_awareness Enable zone awareness for Elasticsearch cluster (default false) bool "false" no
es_zone_awareness_count Number of availability zones used for data nodes (default 2) number "2" no
instance_count Number of data nodes in the cluster (default 6) number "6" no
instance_type ES instance type for data nodes in the cluster (default t2.small.elasticsearch) string "t2.small.elasticsearch" no
kms_key_id KMS key used for elasticsearch string "" no
log_publishing_options List of maps of options for publishing slow logs to CloudWatch Logs. list(map(string)) [] no
management_iam_roles List of IAM role ARNs from which to permit management traffic (default ['*']). Note that a client must match both the IP address and the IAM role patterns in order to be permitted access. list(string) [ "*" ] no
management_public_ip_addresses List of IP addresses from which to permit management traffic (default []). Note that a client must match both the IP address and the IAM role patterns in order to be permitted access. list(string) [] no
node_to_node_encryption_enabled Whether to enable node-to-node encryption. bool "false" no
snapshot_start_hour Hour at which automated snapshots are taken, in UTC (default 0) number "0" no
tags tags to apply to all resources map(string) {} no
tls_security_policy The name of the TLS security policy that needs to be applied to the HTTPS endpoint. Example values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07. Terraform will only perform drift detection if a configuration value is provided. string "null" no
use_prefix Flag indicating whether or not to use the domain_prefix. Default: true bool "true" no
vpc_options A map of supported vpc options map(list(string)) { "security_group_ids": [], "subnet_ids": [] } no

Outputs

Name Description
arn Amazon Resource Name (ARN) of the domain
domain_id Unique identifier for the domain
domain_name The name of the Elasticsearch domain
endpoint Domain-specific endpoint used to submit index, search, and data upload requests
kibana_endpoint Domain-specific endpoint for kibana without https scheme

Authors

Originally created by Steve Huff, Alexander Gramovich and these awesome contributors.

Module managed by Anton Babenko.

License

MIT licensed. See LICENSE.md for full details.