Skip to content

Addressing CNI IP consumption and Warm Pool IP availability calculations #660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Addressing CNI IP consumption and Warm Pool IP availability calculations #660

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion latest/bpg/networking/vpc-cni.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,14 +80,29 @@ You can use the following formula to determine maximum number of Pods you can de

The +2 indicates Pods that require host networking, such as kube-proxy and VPC CNI. Amazon EKS requires kube-proxy and VPC CNI to be operating on each node, and these requirements are factored into the max-pods value. If you want to run additional host networking pods, consider updating the max-pods value. You can specify `--kubelet-extra-args "—max-pods=110"` as user data in the launch template.

As an example, on a cluster with 3 c5.large nodes (3 ENIs and max 10 IPs per ENI), when the cluster starts up and has 2 CoreDNS pods, the CNI will consume 49 IP addresses and keeps them in warm pool. The warm pool enables faster Pod launches when the application is deployed.
As an example, on a cluster with 3 c5.large nodes (3 ENIs and max 10 IPs per ENI), when the cluster starts up and has 2 CoreDNS pods, the CNI will consume 50 IP addresses and keep 43 IPs in warm pool. The warm pool enables faster Pod launches when the application is deployed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought there was a reason this was 49 and not 50?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the logic and explanation in the whole documentation, 49 seems to be incorrect.


Node 1 (with CoreDNS pod): 2 ENIs, 20 IPs assigned

Node 2 (with CoreDNS pod): 2 ENIs, 20 IPs assigned

Node 3 (no Pod): 1 ENI. 10 IPs assigned.

For Node 1 and Node 2 (identical configuration):

- 2 ENIs × 10 IPs per ENI = 20 IPs total
- Subtract 2 primary IPs (1 per ENI) = 18 IPs
- Subtract 1 IP for CoreDNS pod = 17 IPs available
- So each of these nodes has 17 IPs in warm pool

For Node 3:

- 1 ENI × 10 IPs = 10 IPs total
- Subtract 1 primary IP = 9 IPs available in warm pool

Total warm pool calculation:
- 17 (Node 1) + 17 (Node 2) + 9 (Node 3) = 43 IPs

Keep in mind that infrastructure pods, often running as daemon sets, each contribute to the max-pod count. These can include:

* CoreDNS
Expand Down