Difference disableEviction and skipWaitForDeleteTimeout

[adi90x]

Hello,

Is it possible to get example/explanation of the difference between the two ?

Regards,

[dweomer]

From the README:

  # If left unspecified, no drain will be performed.
  # See:
  # - https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/
  # - https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#drain
  drain:
    # deleteLocalData: true  # default
    # ignoreDaemonSets: true # default
    force: true
    # Use `disableEviction == true` and/or `skipWaitForDeleteTimeout > 0` to prevent upgrades from hanging on small clusters.
    # disableEviction: false # default, only available with kubectl >= 1.18
    # skipWaitForDeleteTimeout: 0 # default, only available with kubectl >= 1.18

---

From https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#drain:

Name	Shorthand	Default	Usage
disable-eviction		false	Force drain to use delete, even if eviction is supported. This will bypass checking PodDisruptionBudgets, use with caution.
skip-wait-for-delete-timeout		0	If pod DeletionTimestamp older than N seconds, skip waiting for the pod. Seconds must be greater than 0 to skip.

---

What this means is that prior to 1.18 kubectl drain did not honor Pod Disruption Budgets when draining a node. If you want to replicate this behavior (aka just deleting pods on the draining node) on a 1.18+ cluster with a 1.18+ kubectl you should set your plan .spec.drain.disableEviction to true. If you would prefer to honor PDB up to a maximum-ish number of seconds, then you should set your plan .spec.drain.skipWaitForDeleteTimeout.

If you do not set either of these values for draining nodes in a 1.18+ cluster then, with PDB honored via the Eviction API, your drain operations can hang indefinitely while waiting for an appropriate node to reschedule your pod(s) to. An example of this would be attempting to upgrade a 3-server HA cluster with Rancher 2.x and Longhorn installed: by default Rancher deploys a 3 member ReplicaSet and Longhorn requires 3 members DaemonSets for quorum which means that attempting to perform an upgrade on any of these nodes for a plan with .spec.drain configured that neither disables use of the Eviction API nor sets a deletion timeout will fail to complete successfully.

[dweomer]

@adi90x if you have suggestion(s) for how this can be made more clear in the README I would welcome a pull request!