Skip to content

Commit

Permalink
fix(DMVP-5592): set default karpenter node size to 100Gi
Browse files Browse the repository at this point in the history
  • Loading branch information
mrdntgrn committed Dec 9, 2024
1 parent babd387 commit 757b315
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 3 deletions.
2 changes: 1 addition & 1 deletion modules/karpenter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ module "karpenter" {
| <a name="input_oidc_provider_arn"></a> [oidc\_provider\_arn](#input\_oidc\_provider\_arn) | EKC oidc provider arn in format 'arn:aws:iam::<account-id>:oidc-provider/oidc.eks.<region>.amazonaws.com/id/<oidc-id>'. | `string` | n/a | yes |
| <a name="input_resource_chart_version"></a> [resource\_chart\_version](#input\_resource\_chart\_version) | The dasmeta karpenter-resources chart version | `string` | `"0.1.0"` | no |
| <a name="input_resource_configs"></a> [resource\_configs](#input\_resource\_configs) | Configurations to pass and override default ones for karpenter-resources chart. Check the helm chart available configs here: https://github.com/dasmeta/helm/tree/karpenter-resources-0.1.0/charts/karpenter-resources | `any` | `{}` | no |
| <a name="input_resource_configs_defaults"></a> [resource\_configs\_defaults](#input\_resource\_configs\_defaults) | Configurations to pass and override default ones for karpenter-resources chart. Check the helm chart available configs here: https://github.com/dasmeta/helm/tree/karpenter-resources-0.1.0/charts/karpenter-resources | <pre>object({<br> nodeClass = optional(any, {<br> amiFamily = "AL2" # Amazon Linux 2<br> detailedMonitoring = true<br> metadataOptions = {<br> httpEndpoint = "enabled"<br> httpProtocolIPv6 = "disabled"<br> httpPutResponseHopLimit = 2 # This is changed to disable IMDS access from containers not on the host network<br> httpTokens = "required"<br> }<br> })<br> nodeClassRef = optional(any, {<br> group = "karpenter.k8s.aws"<br> kind = "EC2NodeClass"<br> name = "default"<br> }),<br> requirements = optional(any, [<br> {<br> key = "karpenter.k8s.aws/instance-cpu"<br> operator = "Lt"<br> values = ["9"] # <=8 core cpu nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-memory"<br> operator = "Lt"<br> values = ["33000"] # <=32 Gb memory nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-memory"<br> operator = "Gt"<br> values = ["1000"] # >1Gb Gb memory nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-generation"<br> operator = "Gt"<br> values = ["2"] # generation of ec2 instances >2 (like t3a.medium) are more performance and effectiveness<br> },<br> {<br> key = "kubernetes.io/arch"<br> operator = "In"<br> values = ["amd64"] # amd64 linux is main platform arch we will use<br> },<br> {<br> key = "karpenter.sh/capacity-type"<br> operator = "In"<br> values = ["spot", "on-demand"] # both spot and on-demand nodes, it will look at first available spot and if no then on-demand<br> }<br> ])<br> disruption = optional(any, {<br> consolidationPolicy = "WhenEmptyOrUnderutilized"<br> consolidateAfter = "1m"<br> }),<br> limits = optional(any, {<br> cpu = 10<br> })<br> })</pre> | `{}` | no |
| <a name="input_resource_configs_defaults"></a> [resource\_configs\_defaults](#input\_resource\_configs\_defaults) | Configurations to pass and override default ones for karpenter-resources chart. Check the helm chart available configs here: https://github.com/dasmeta/helm/tree/karpenter-resources-0.1.0/charts/karpenter-resources | <pre>object({<br> nodeClass = optional(any, {<br> amiFamily = "AL2" # Amazon Linux 2<br> detailedMonitoring = true<br> metadataOptions = {<br> httpEndpoint = "enabled"<br> httpProtocolIPv6 = "disabled"<br> httpPutResponseHopLimit = 2 # This is changed to disable IMDS access from containers not on the host network<br> httpTokens = "required"<br> }<br> blockDeviceMappings = [<br> {<br> deviceName = "/dev/xvda"<br> ebs = {<br> volumeSize = "100Gi"<br> volumeType = "gp3"<br> encrypted = true<br> }<br> }<br> ]<br> })<br> nodeClassRef = optional(any, {<br> group = "karpenter.k8s.aws"<br> kind = "EC2NodeClass"<br> name = "default"<br> }),<br> requirements = optional(any, [<br> {<br> key = "karpenter.k8s.aws/instance-cpu"<br> operator = "Lt"<br> values = ["9"] # <=8 core cpu nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-memory"<br> operator = "Lt"<br> values = ["33000"] # <=32 Gb memory nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-memory"<br> operator = "Gt"<br> values = ["1000"] # >1Gb Gb memory nodes<br> },<br> {<br> key = "karpenter.k8s.aws/instance-generation"<br> operator = "Gt"<br> values = ["2"] # generation of ec2 instances >2 (like t3a.medium) are more performance and effectiveness<br> },<br> {<br> key = "kubernetes.io/arch"<br> operator = "In"<br> values = ["amd64"] # amd64 linux is main platform arch we will use<br> },<br> {<br> key = "karpenter.sh/capacity-type"<br> operator = "In"<br> values = ["spot", "on-demand"] # both spot and on-demand nodes, it will look at first available spot and if no then on-demand<br> }<br> ])<br> disruption = optional(any, {<br> consolidationPolicy = "WhenEmptyOrUnderutilized"<br> consolidateAfter = "1m"<br> }),<br> limits = optional(any, {<br> cpu = 10<br> })<br> })</pre> | `{}` | no |
| <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | VPC subnet ids used for default Ec2NodeClass as subnet selector. | `list(string)` | n/a | yes |
| <a name="input_wait"></a> [wait](#input\_wait) | Whether use helm deploy with --wait flag | `bool` | `true` | no |

Expand Down
5 changes: 3 additions & 2 deletions modules/karpenter/locals.tf
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ locals {
amiSelectorTerms = [
{ id = data.aws_instance.ec2_from_eks_node_pool.ami }
]
detailedMonitoring = var.resource_configs_defaults.nodeClass.detailedMonitoring
metadataOptions = var.resource_configs_defaults.nodeClass.metadataOptions
detailedMonitoring = var.resource_configs_defaults.nodeClass.detailedMonitoring
metadataOptions = var.resource_configs_defaults.nodeClass.metadataOptions
blockDeviceMappings = var.resource_configs_defaults.nodeClass.blockDeviceMappings
}

nodePoolDefaultNodeClassRef = var.resource_configs_defaults.nodeClassRef
Expand Down
10 changes: 10 additions & 0 deletions modules/karpenter/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,16 @@ variable "resource_configs_defaults" {
httpPutResponseHopLimit = 2 # This is changed to disable IMDS access from containers not on the host network
httpTokens = "required"
}
blockDeviceMappings = [
{
deviceName = "/dev/xvda"
ebs = {
volumeSize = "100Gi"
volumeType = "gp3"
encrypted = true
}
}
]
})
nodeClassRef = optional(any, {
group = "karpenter.k8s.aws"
Expand Down

0 comments on commit 757b315

Please sign in to comment.