Skip to content
This repository was archived by the owner on Jan 9, 2020. It is now read-only.

Commit 7f14b71

Browse files
committed
Added a comment regarding how failed executor pods are handled
1 parent 0ab9ca7 commit 7f14b71

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -153,9 +153,12 @@ private[spark] class KubernetesClusterSchedulerBackend(
153153
} { executorExited =>
154154
logWarning(s"Removing executor $executorId with loss reason " + executorExited.message)
155155
removeExecutor(executorId, executorExited)
156-
// We keep around executors that have exit conditions caused by the application. This
157-
// allows them to be debugged later on. Otherwise, mark them as to be deleted from the
158-
// the API server.
156+
// We don't delete the pod running the executor that has an exit condition caused by
157+
// the application from the Kubernetes API server. This allows users to debug later on
158+
// through commands such as "kubectl logs <pod name>" and
159+
// "kubectl describe pod <pod name>". Note that exited containers have terminated and
160+
// therefore won't take CPU and memory resources.
161+
// Otherwise, the executor pod is marked to be deleted from the API server.
159162
if (executorExited.exitCausedByApp) {
160163
logInfo(s"Executor $executorId exited because of the application.")
161164
deleteExecutorFromDataStructures(executorId)

0 commit comments

Comments
 (0)