Tasks should be created with task group ID of decision task

[petemoore]

Currently taskgraph assumes that taskId and taskGroupId match. This means, if you retrigger a decision task, but e.g. place it in the original task group, subtasks will end up in a completely different task group to the decision task. Or maybe you want to have more than one decision task in your task group. To ensure that subtasks are always created in same task group as decision task, the subtask should have its taskGroupId set to the taskGroupId of the decision task (not the taskId of the decision task).

One option would be to query the taskcluster queue to get the taskGroupId of the current task. Another option would be to check if TASK_GROUP_ID is set in the environment, and fall back to querying taskcluster queue if it isn't. The reason for this, is that TASK_GROUP_ID env var has been added to Generic Worker tasks in taskcluster/taskcluster#7140, so should be available in future to taskgraph.

Code is here:

taskgraph/src/taskgraph/create.py

Line 45 in 4896be1

task_def["taskGroupId"] = decision_task_id

[jcristau]

if you retrigger a decision task, but e.g. place it in the original task group

Why is this something we'd want to do/support?

[petemoore]

if you retrigger a decision task, but e.g. place it in the original task group

Why is this something we'd want to do/support?

We already support it in the sense that the Retrigger feature in the taskcluster UI will create a new task with the same task group ID but with a different task ID.

I think the question is though, if a task that calls taskgraph has a different Task ID to its Task Group ID, which should be used for the tasks that taskgraph creates? I can't see a reason taskgraph should create subtasks in a different task group, so I would argue it should use Task Group ID and not Task ID. If we don't want to support tasks running in taskgraph that don't share Task ID and Task Group ID, taskgraph should probably check for that condition and fail if they do not match. In either case, it would need to look up the Task Group ID.