KAN-222 Add Redis Failover connection testing with Lettuce (#105)

backend/README.md

@@ -0,0 +1,2 @@
+# 목차
+* [Lettuce 라이브러리에서 Redis Failover connection 테스트](./springboot_lettuce_connection/README.md)

backend/springboot_lettuce_connection/README.md

@@ -0,0 +1,21 @@
+# 개요
+* Lettuce 라이브러리에서 Redis Failover connection 테스트
+
+# 목차
+* [springboot 소스코드](./connection-test/)
+* 실습환경 구축
+  * [kind cluster](./kind-cluster/)
+  * [Redis cluster](./manifests/redis-cluster/)
+  * [kubernetes manifest](./manifests/)
+  * [옵션 ArgoCD](../../common_k8s_manifests/argocd/use_helm/)
+  * [옵션localstack](./localstack/)
+  * [옵션 EKS](./terraform/)
+
+# 참고자료
+* https://docs.google.com/presentation/d/1FtEFBCubpcqMJ6C7YV55KAxjhZ5znYn6A3f1c341Lcg/edit#slide=id.g25a881cd3a5_0_166
+* https://jojoldu.tistory.com/418
+* https://github.com/redis/lettuce/wiki/Connection-Pooling
+* https://learn.microsoft.com/ko-kr/azure/azure-cache-for-redis/cache-best-practices-connection
+* https://velog.io/@komment/Spring-Boot-Redis-Cluster-with-lettuce-redisson
+* https://iizz.tistory.com/200
+* https://blog.leocat.kr/notes/2022/04/15/lettuce-config-for-redis-cluster-topology-refresh

backend/springboot_lettuce_connection/kind-cluster/Makefile

@@ -0,0 +1,5 @@
+up:
+	kind create cluster --config kind-config.yaml
+
+down:
+	kind delete cluster --name redis-example

backend/springboot_lettuce_connection/kind-cluster/README.md

@@ -0,0 +1,14 @@
+# 개요
+* kind cluster에 redis cluster 생성
+
+# kind 클러스터 생성
+
+```sh
+make up
+```
+
+# kind 클러스터 삭제
+
+```sh
+make down
+```

backend/springboot_lettuce_connection/kind-cluster/kind-config.yaml

@@ -0,0 +1,21 @@
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+name: redis-example
+nodes:
+- role: control-plane
+  image: kindest/node:v1.30.4
+  extraPortMappings:
+  # springboot nodePort
+  - containerPort: 32101
+    hostPort: 32101
+    protocol: TCP
+  # ArgoCD HTTPS nodePort
+  - containerPort: 30443
+    hostPort: 30443
+    protocol: TCP
+- role: worker
+  image: kindest/node:v1.30.4
+- role: worker
+  image: kindest/node:v1.30.4
+- role: worker
+  image: kindest/node:v1.30.4

backend/springboot_lettuce_connection/localstack/README.md

@@ -0,0 +1,38 @@
+# 개요
+* 로컬환경에서 ElasticCache 구축
+
+# 준비
+* localstack
+* Localstack pro 라이센스
+
+# 생성방법
+
+* ElasticCache 클러스터 생성
+
+```sh
+awslocal elasticache create-replication-group \
+  --engine redis \
+  --replication-group-id my-clustered-redis-replication-group \
+  --replication-group-description 'my clustered replication group' \
+  --cache-node-type cache.t2.micro \
+  --num-node-groups 2 \
+  --replicas-per-node-group 2
+```
+
+* 클러스터 엔드포인트 조회
+
+```sh
+awslocal elasticache describe-replication-groups --replication-group-id my-clustered-redis-replication-group \
+    --query "ReplicationGroups[0].ConfigurationEndpoint"
+{
+  "Address": "localhost.localstack.cloud",
+  "Port": 4510
+}
+```
+
+# 삭제 방법
+
+```sh
+awslocal elasticache delete-replication-group \
+  --replication-group-id my-clustered-redis-replication-group
+```

backend/springboot_lettuce_connection/manifests/argocd_bootstrap/README.md

@@ -0,0 +1,22 @@
+# 개요
+* springboot lettuce 라이브러리 장애 재현을 위한 ArgoCD Application bootstrap
+
+# 전제조건
+* ArgoCD가 설치되어 있어야 함
+
+# 목차
+* [ArgoCD 설치 후 초기 설정](./initialize.yaml)
+* [springboot application](./springboot_application.yaml)
+* [redis-cluster](./redis-cluster.yaml)
+
+# 생성 방법
+
+```sh
+kubectl apply -f ./
+```
+
+# 삭제 방법
+
+```sh
+kubectl delete -f ./
+```

backend/springboot_lettuce_connection/manifests/argocd_bootstrap/redis-cluster.yaml

@@ -0,0 +1,21 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: redis
+  namespace: argocd
+spec:
+  project: default
+  destination:
+    namespace: default
+    server: https://kubernetes.default.svc
+  source:
+    repoURL: oci://registry-1.docker.io/bitnamicharts
+    targetRevision: latest
+    chart: redis-cluster
+    helm:
+      values: |
+        # primary instance만 사용
+        cluster:
+          nodes: 3
+          replicas: 0
+        usePassword: false

backend/springboot_lettuce_connection/manifests/argocd_bootstrap/springboot_application.yaml

@@ -0,0 +1,46 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: springboot-lettuce-connection
+  namespace: argocd
+spec:
+  project: default
+  destination:
+    namespace: default
+    server: https://kubernetes.default.svc
+  source:
+    repoURL: https://build-deploy-pipeline.github.io/helm-charts/
+    targetRevision: 0.1.0
+    chart: lettuce-connection
+    helm:
+      values: |
+        image:
+          repository: choisunguk/lettuce-connection
+          tag: v1 # lettuce 안전성 오류가 있는 버전
+          # tag: v2 # 디버깅 로그 추가 버전
+          # tag: v3 # 오류 해결 버전
+        # use node port on kind-cluster
+        # service:
+        #   type: NodePort
+        #   nodePort: 32101
+        resources:
+          limits:
+            memory: 256Mi
+          requests:
+            cpu: 300m
+            memory: 256Mi
+        env:
+        - name: REDIS_HOST
+          # use on kind-cluster
+          # value: "redis-redis-cluster.default.svc.cluster.local"
+          # use on ElasticCache
+          value: "redis-cluster.ti9epm.clustercfg.apn2.cache.amazonaws.com"
+        - name: REDIS_PORT
+          value: "6379"
+  syncPolicy:
+    syncOptions:
+    - CreateNamespace=true
+    - ApplyOutOfSyncOnly=true
+    automated:
+      prune: true
+      selfHeal: true

backend/springboot_lettuce_connection/manifests/lettuce_springboot/service.yaml

@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: lettuce-connection
+spec:
+  selector:
+    app: lettuce-connection
+  ports:
+  - port: 80
+    targetPort: 8080
+    nodePort: 32101
+  type: NodePort

backend/springboot_lettuce_connection/manifests/lettuce_springboot/springboot_deployment.yaml

@@ -0,0 +1,37 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: lettuce-connection
+  namespace: default
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: lettuce-connection
+  template:
+    metadata:
+      labels:
+        app: lettuce-connection
+    spec:
+      containers:
+      - name: springboot
+        # 안전성 오류가 있는 버전
+        image: choisunguk/lettuce-connection:v1
+        # 디버깅 로그 추가 버전
+        # image: choisunguk/lettuce-connection:v2
+        # 오류 해결 버전
+        # image: choisunguk/lettuce-connection:v3
+        imagePullPolicy: IfNotPresent
+        env:
+        - name: REDIS_HOST
+          value: "redis-redis-cluster.default.svc.cluster.local"
+        - name: REDIS_PORT
+          value: "6379"
+        ports:
+        - containerPort: 8080
+        resources:
+          limits:
+            memory: 256Mi
+          requests:
+            cpu: 300m
+            memory: 256Mi

backend/springboot_lettuce_connection/manifests/redis-cluster/README.md

@@ -0,0 +1,12 @@
+# 개요
+* redis-cluster 생성
+
+# 생성 방법
+
+```sh
+helm upgrade --install redis oci://registry-1.docker.io/bitnamicharts/redis-cluster \
+  -n default \
+  --set cluster.nodes=3 \
+  --set cluster.replicas=0 \
+  --set usePassword=false
+```

backend/springboot_lettuce_connection/terraform/Readme.md

@@ -0,0 +1,64 @@
+# 개요
+* 테라폼으로 VPC, EKS, ElasticCache 생성
+
+<br>
+
+# 생성 방법
+
+* 테라폼 변수를 환경변수로 설정
+
+```bash
+# AWS profile
+export TF_VAR_assume_role_arn=""
+```
+
+* 테라폼 코드 실행
+```bash
+terraform init
+terraform plan
+terraform apply # 약 15~20분 소요
+````
+
+* kubeconfig 생성
+
+```bash
+# kubeconfig 생성
+aws eks update-kubeconfig --region ap-northeast-2 --name eks-from-terraform
+
+# cluster 확인
+kubectl cluster-info
+```
+
+<br>
+
+# (옵션) Amazon prometheus를 사용하여 EKS 메트릭 수집
+
+1. 테라폼 변수에서 enable_amp를 true로 설정
+2. terraform apply(약 20분 소요)
+3. [문서](./Amazon_prometheus.md)를 참고하여 grafana<->Amazon proemtheus 연동
+
+# 삭제 방법
+
+```bash
+terrform destroy
+```
+
+<br>
+
+# 참고자료
+* terraform module 디버깅: https://thoeny.dev/how-to-debug-in-terraform
+* terraform splat: https://developer.hashicorp.com/terraform/language/expressions/splat
+* terraform eks overview: https://www.linkedin.com/pulse/eks-cluster-aws-day21-vijayabalan-balakrishnan/
+* terraform eks overview: https://dev.to/aws-builders/how-to-build-eks-with-terraform-54pl
+* terraform eks overview: https://devpress.csdn.net/cicd/62ec845619c509286f4172fc.html
+* terraform eks additional security group: https://saturncloud.io/blog/terraform-additional-security-group-for-managed-nodes-in-eks-a-comprehensive-guide/
+* terraform eks aws-auth configmap: https://medium.com/@codingmaths/aws-eks-cluster-with-terraform-ebf0d2583f9a
+* terraform eks aws-auth configmap: https://dev.to/fukubaka0825/manage-eks-aws-auth-configmap-with-terraform-4ndp
+* terraform eks aws-auth configmap: https://github.com/cloudposse/terraform-aws-eks-cluster/blob/main/auth.tf
+* terraform eks aws-auth IAM role: https://medium.com/@radha.sable25/enabling-iam-users-roles-access-on-amazon-eks-cluster-f69b485c674f
+* terraform eks addons: https://dev.to/aws-builders/install-manage-amazon-eks-add-ons-with-terraform-2dea
+* terraform irsa: https://medium.com/@tech_18484/step-by-step-guide-creating-an-eks-cluster-with-terraform-resources-iam-roles-for-service-df1c5e389811
+* terraform aws_iam_policy_document: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment
+* migrate from aws-auth to access entry: https://fixit-xdu.medium.com/aws-eks-access-entry-4a7e25ed6c3a
+* migrate from aws-auth to access entry: https://opsinsights.dev/simplifying-access-entries-in-eks-a-guide
+* aws docs - accesss entry: https://docs.aws.amazon.com/eks/latest/userguide/migrating-access-entries.html

backend/springboot_lettuce_connection/terraform/eks.tf

@@ -0,0 +1,47 @@
+module "eks" {
+  source = "git::https://github.com/choisungwook/terraform_practice.git//eks/module/eks?ref=0.1"
+
+  eks_cluster_name      = var.eks_cluster_name
+  eks_version           = var.eks_version
+  oidc_provider_enabled = var.oidc_provider_enabled
+
+  vpc_id                  = module.vpc.vpc_id
+  private_subnets_ids     = module.vpc.private_subnets_ids
+  endpoint_private_access = var.endpoint_private_access
+  endpoint_public_access  = var.endpoint_public_access
+
+  # 아래 명령어를 실행하여 addon version을 설정하세요
+  # aws eks describe-addon-versions --kubernetes-version {eks_verison} --addon-name {addon_name} --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table
+  eks_addons = [
+    # https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/managing-kube-proxy.html
+    {
+      name                 = "kube-proxy"
+      version              = "v1.29.0-eksbuild.3"
+      configuration_values = jsonencode({})
+    },
+    # https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/managing-vpc-cni.html
+    {
+      name                 = "vpc-cni"
+      version              = "v1.16.2-eksbuild.1"
+      configuration_values = jsonencode({})
+    },
+    # https://docs.aws.amazon.com/ko_kr/eks/latest/userguide/managing-coredns.html
+    {
+      name                 = "coredns"
+      version              = "v1.11.1-eksbuild.6"
+      configuration_values = jsonencode({})
+    }
+  ]
+
+  managed_node_groups = var.managed_node_groups
+
+  // IRSA role 생성 여부
+  karpenter_enabled      = true
+  alb_controller_enabled = true
+  external_dns_enabled   = true
+
+  // EKS access entry 설정
+  aws_auth_admin_roles = [
+    var.assume_role_arn
+  ]
+}