Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] AKS API server vnet integration failed #4783

Open
CreMindES opened this issue Feb 9, 2025 · 2 comments
Open

[BUG] AKS API server vnet integration failed #4783

CreMindES opened this issue Feb 9, 2025 · 2 comments
Assignees

Comments

@CreMindES
Copy link

CreMindES commented Feb 9, 2025

Describe the bug
AKS API server vnet integration failed. It took ~60 minutes to fail. It seems that the api server is not able to communicate with the nodes.

To Reproduce
Steps to reproduce the behavior:

  1. Have a public AKS cluster
  2. Have at least one workload with a strict pod disruption policy, e.g. loki
  3. Create a subnet for the api server in the vnet of AKS
  4. Run command az aks update --name <cluster-name> --resource-group <resource-group> --enable-apiserver-vnet-integration --apiserver-subnet-id <apiserver-subnet-resource-id> as per docs.
  5. Wait ~60+ minutes.
  6. See error
    Code: UpgradeFailed 
    Message: Drain node aks-..... failed when evicting pod loki-backend-0 failed with Too Many Requests error.   
    This is often caused by a restrictive Pod Disruption Budget (PDB) policy. See http://aka.ms/aks/debugdrainfailures. Original error: Cannot 
    evict pod as it would violate the pod's disruption budget.. PDB debug info: ns/loki-backend-0 blocked by pdb loki-backend 
    (MaxUnavailable: 1) with 2 unready pods: [observability/loki-backend-0,observability/loki-backend-1].
    

Expected behavior
AKS VNet Integration enable works and finishes in a <10 minutes.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • CLI Version 2.38
  • Kubernetes version 1.30.3
  • CLI Extension version [email protected]
  • Browser [e.g. chrome, safari] is applicable

Additional context
Add any other context about the problem here.

@CreMindES CreMindES added the bug label Feb 9, 2025
@CreMindES CreMindES changed the title [BUG] AKS API server vnet integration failed due to pod eviction failure [BUG] AKS API server vnet integration failed Feb 9, 2025
Copy link
Contributor

@chasewilson would you be able to assist?

@feiskyer
Copy link
Member

From the error message, it is clear the PDB blocked upgrade process. @CreMindES could you fix the PDB settings and retry again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants