Skip to content
This repository has been archived by the owner on Jul 22, 2024. It is now read-only.

draining stuck on hard instance failure #8

Open
svenwltr opened this issue Sep 24, 2018 · 0 comments
Open

draining stuck on hard instance failure #8

svenwltr opened this issue Sep 24, 2018 · 0 comments
Labels
bug Something isn't working

Comments

@svenwltr
Copy link
Member

svenwltr commented Sep 24, 2018

The node-drainer gets stuck on a hard instance failure. In this case:

  • the node gets NotReady in Kubernetes
  • the pods on this node get either Unknown or NodeLost
  • the instance is in Terminating:Wait state

It gets stuck, because:

  1. The node gets not removed from Kubernetes, because it is still in the ASG.
  2. The instance does not get removed from the ASG, because it waits for the draining.
  3. The draining cannot happen, because the node is actually gone.

Also the node is nor reachable via SSH nor ping.

@svenwltr svenwltr added the bug Something isn't working label Sep 24, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant