Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add restart for namespace workloads. #79

Merged
merged 4 commits into from
Feb 6, 2024
Merged

Conversation

jefchien
Copy link
Contributor

@jefchien jefchien commented Feb 5, 2024

Issue #, if available: N/A

Description of changes:
Based on restart logic from kubectl (https://github.com/kubernetes/kubectl/blob/master/pkg/polymorphichelpers/objectrestarter.go#L32). When a namespace is changed, it restarts all deployments, daemonsets, and statefulsets within the namespace by setting an annotation and updating the resource.

Testing:

Create the namespace/deployments (taken from https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#creating-a-deployment)

> kubectl create ns coffee-shop
> kubectl apply -f deployment.yaml -n coffee-shop
> kubectl describe deployment/nginx -n coffee-shop

Name:                   nginx
Namespace:              coffee-shop
CreationTimestamp:      Mon, 05 Feb 2024 10:37:09 -0500
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:1.14.2
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-7fb96c846b (3/3 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  6m52s  deployment-controller  Scaled up replica set nginx-7fb96c846b to 3

Update auto-annotation config, which recreates the pod in the deployment.

> kubectl set env deployment/cloudwatch-controller-manager AUTO_ANNOTATION_CONFIG="{\"java\":{\"namespaces\":[\"coffee-shop\"]}}" -n amazon-cloudwatch

This annotates the namespace and restarts the deployments in it.

> kubectl describe deployment/nginx -n coffee-shop

Name:                   nginx
Namespace:              coffee-shop
CreationTimestamp:      Mon, 05 Feb 2024 10:37:09 -0500
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 2
Selector:               app=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:       app=nginx
  Annotations:  cloudwatch.aws.amazon.com/restartedAt: 2024-02-05T15:53:40Z
  Containers:
   nginx:
    Image:        nginx:1.14.2
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  nginx-7fb96c846b (0/0 replicas created)
NewReplicaSet:   nginx-58c9965b4b (3/3 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  17m   deployment-controller  Scaled up replica set nginx-7fb96c846b to 3
  Normal  ScalingReplicaSet  31s   deployment-controller  Scaled up replica set nginx-58c9965b4b to 1
  Normal  ScalingReplicaSet  19s   deployment-controller  Scaled down replica set nginx-7fb96c846b to 2 from 3
  Normal  ScalingReplicaSet  19s   deployment-controller  Scaled up replica set nginx-58c9965b4b to 2 from 1
  Normal  ScalingReplicaSet  17s   deployment-controller  Scaled down replica set nginx-7fb96c846b to 1 from 2
  Normal  ScalingReplicaSet  17s   deployment-controller  Scaled up replica set nginx-58c9965b4b to 3 from 2
  Normal  ScalingReplicaSet  14s   deployment-controller  Scaled down replica set nginx-7fb96c846b to 0 from 1

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

go.mod Show resolved Hide resolved
Renamed restart annotation.
okankoAMZ
okankoAMZ previously approved these changes Feb 6, 2024
Copy link
Contributor

@okankoAMZ okankoAMZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!, left few comments

main.go Show resolved Hide resolved
pkg/instrumentation/auto/annotation.go Show resolved Hide resolved
pkg/instrumentation/auto/restart.go Show resolved Hide resolved
@jefchien jefchien merged commit 0503ef6 into aws:main Feb 6, 2024
2 checks passed
@jefchien jefchien deleted the auto-restart branch February 6, 2024 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants