Skip to content

Latest commit

 

History

History
87 lines (63 loc) · 3.77 KB

ci-concept.md

File metadata and controls

87 lines (63 loc) · 3.77 KB
Raw

CI with Tekton

This document presents an overview of how to do CI on Tekton using Tekton itself. The same concept can be ported to other applications.

Pipeline and Triggers resources

We use a dedicated Event Listener to serve webhooks from repo for which we want to run CI. The Event Listener filters out invalid requests and unwanted events using the stock GitHub interceptor. CEL filters can then be used to select which events we want to react to by triggering the CI pipelines.

Since not all different types of events include the same information in the same place, trigger bindings and custom interceptors with overlays are used to present a standard interface towards the trigger bindings.

We use multiple bindings to make them re-usable:

  • A GitHub base binding extracts parameters common to all GitHub events
  • GitHub event specific bindings extract other parameters

Custom interceptors are used to add more information into the payload about the Pull Request and the user who submitted it.

Trigger templates receive parameters from the bindings and run the CI pipelines. They are organized per repo:

  • One trigger template includes all CI jobs common to all repos
  • Each repo then has an own trigger template, which can be hosted in the repo itself.

Trigger templates may include tasks and pipelines that are not CI jobs that may perform tasks required for the actual CI jobs to work correctly.

CI Jobs are implemented as pipelines with one or more tasks and one or more conditions. Conditions are used to ensure only relevant CI jobs are executed:

  • Only run a CI job when relevant files were modified
  • Only run a CI job when the job name matches the regex requested via the /test [regex] command in GitHub

The Pipeline is used to adapt the input parameters available in the trigger template to those needed by the CI Task(s). It adds labels and annotations useful for identifying CI Jobs and for GitHub update logic downstream.

CI Diagram

GitHub check updates

Tekton for CI is configured to send cloud events notifications for all TaskRun and PipelineRun life-cycle events. The events are sent to a dedicated tekton-events event listener, which gathers and filters them, and uses them to update checks on GitHub side.

When a CI Task starts, the start event is intercepted and the GitHub check is set to pending, with a link to the Tekton Dashboard for developers to stream execution logs live if they wish to.

When a CI Task completes, the end event is intercepted to the GitHub check is set to passed or failed, depending on the outcome of the Task.

If a CI job is made of multiple Tasks, extra filtering logic is required to select the correct events for GitHub status updates:

  • Pipeline Start -> We never use this one as condition may tell us later that the CI job should not be executed
  • Task Start -> This can be used to set the status to pending, even in case of multiple tasks
  • Task End -> This can be used in case of single task CI jobs, or also in case of multiple tasks if we have annotations to tell us that it's the last one
  • Pipeline End -> This can used in all cases, however the Task End is preferred in case of single Task CI jobs since it will trigger faster

In edge cases events may be delivered out of order (for instance in case of retries) which could lead to the start event being processed after the end one. Logic in the GitHub update tasks helps minimize the impact of this case.

Reuse of Tasks in CI

It is highly desirable to be able to use existing Tasks in CI, for instance Tasks from the catalog to run unit tests or linting. When a suitable Task does not exists, it should be added to the catalog if applicable.

The CI pipelines can be used to adapt the parameters available from the CI system to the catalog tasks.