You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the ASR task in Airflow fails every so often when the Whisper service gets evicted or OOMKilled. The triggerer will inquire about the status of a task ID that does not exist in the newly started Whisper service pod.
It would be nice if we can prevent this somehow, so that the same resources doesn't have to be processed for ASR again.
We should think about a good way to go about this. One proposal that we might discuss:
On receiving sigterm, stay alive until the task has finished AND Airflow trigger has been informed (that is: the API has sent a Status.DONE: status.HTTP_200_OK response upon a @api.get("/tasks/{task_id}") request.
The text was updated successfully, but these errors were encountered:
Currently, the ASR task in Airflow fails every so often when the Whisper service gets evicted or OOMKilled. The triggerer will inquire about the status of a task ID that does not exist in the newly started Whisper service pod.
It would be nice if we can prevent this somehow, so that the same resources doesn't have to be processed for ASR again.
We should think about a good way to go about this. One proposal that we might discuss:
On receiving sigterm, stay alive until the task has finished AND Airflow trigger has been informed (that is: the API has sent a
Status.DONE: status.HTTP_200_OK
response upon a@api.get("/tasks/{task_id}")
request.The text was updated successfully, but these errors were encountered: