Add TES idempotency feature #707
Labels
enhancement
New feature or request
Performance
Enable users can run task as cheap and as fast as possible
Scalability
Enable users can scale TES workloads
TES Priority: P2
Groomed to a Priority 2 issue
If
EnabledWithOutputCopying
, then TES shall use Azure server-side blob copy to copy the previous task's outputs to the current task output's specified location(s). This work item should be added to an in-memory queue and the task state shall be set toRUNNING
. It should be done in a non-blocking way from the main task status checking loop, so as not to slow down overall task throughput (Tasks can have thousands of files that need to be copied, and even though it's done server side, calling that API 1000 times will take a while). Before starting the copy, the task state shall be set toRUNNING
. There shall be two separate C# HostedServices that are long-running (Created instartup.cs
), one and periodically checking if all of the copies are complete; then set the task state toCOMPLETE
. The other should be checking if any blob copy on the file(s) is already in progress, and if not, start the copy. If TES crashes, it should be able to pickup where it left off by looping through allRUNNING
tasks and resuming each one that is currently copying inputs.The text was updated successfully, but these errors were encountered: