Merge pull request #141 from Roming22/adr-0009

ADR-0009: Pipeline Service via Operator

ADR/0001-pipeline-service-phase-1.md

@@ -1,11 +1,13 @@
 # 1. Pipeline Service Phase 1
 
 Created: 2022-10-13
-Last Updated: 2022-10-13
+Last Updated: 2023-09-29
 
 ## Status
 
-Accepted
+Obsolete
+
+Superceded by [ADR-0009](./0009-pipeline-service-via-operator.md)
 
 ## Context
 

ADR/0009-pipeline-service-via-operator.md

@@ -0,0 +1,74 @@
+# 0009 Pipeline Service via Operator
+
+Created: 2023-09-29
+Last Updated: 2023-09-29
+
+## Status
+
+Proposed
+
+## Context
+
+kcp is no longer being used as a control plane for RHTAP. This means that
+"Pipeline Service" cannot be deployed as an independent service. For our
+initial MVP, all Tekton APIs need to be deployed onto a standard OpenShift
+cluster (specifically OpenShift Dedicated).
+
+## Decision
+
+All Tekton APIs will be provided using the stock OpenShift Pipelines operator (OSP).
+In the spirit of developing in a "Service First" manner, RHTAP will deploy
+a candidate "nightly" release of the operator. The service will be defined in
+the [pipeline-service](https://github.com/openshift/pipeline-service)
+repository, which is then imported into
+[infra-deployments](https://github.com/redhat-appstudio/infra-deployments) as
+an ArgoCD application.
+
+Not all metrics required for operating the service are exposed natively by the
+controllers. The `pipeline-metrics-exporter` controller is to be used as a test
+bed to expose new metrics, with the goal of upstreaming those metrics as they
+mature and prove their value.
+
+Configurations that are specific to RHTAP must be made available through
+the OSP CRDs. The following changes are specific to RHTAP:
+
+- Disable Tekton Triggers. Pipelines as Code will be the favored mechanism for
+  event-based triggering of pipelines for now. This decision can be revisited
+  in the future based on need.
+- Disable the pruner that comes with the operator. Tekton Results will be used
+  to prune `PipelineRun` and `TaskRun` data off cluster, thereby ensuring data
+  is archived before removal.
+- Direct Pipelines as Code to use a URL pattern that displays the `PipelineRun`
+  or `TaskRun` info from the Hybrid Application Console (HAC). This ensures
+  end users do not need access to the underlying compute cluster(s).
+- The Pipelines as Code application name must match the GitHub Application name, so that users understand which GitHubApplication is responsible for triggering the pipelines.
+- The GitHub Application secret value, deployed using an ExternalSecret.
+- Any configuration related to performance.
+
+Furthermore, as the service will be accessed through CodeReadyToolchain (CRT), the
+following changes are also specific to RHTAP:
+- Deploying a proxy (known as `SprayProxy`) on the CRT host cluster that redirects
+  incoming PaC requests to the member clusters. Requests are redirected only if:
+  - the payload is validated using the webhook secret;
+  - the payload size is under 25MB.
+- Providing a plugin to the CRT Proxy so Tekton Results requests are redirected
+  to the appropriate member cluster.
+
+## Consequences
+
+- Tekton Triggers should be disabled in RHTAP using the appropriate operator
+  configuration.
+- The Tekton Pruner needs to be disabled in RHTAP using the appropriate
+  operator configuration. This is done under the assumption that Results will
+  be responsible for pruning resources. Eventually the operator should automate
+  this setting if `Results` is deployed and configured to prune resources.
+- Pipelines as Code should use an appropriate URL to HAC when interacting with
+  SCM services, such as the GitHub
+  [Checks API](https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api?apiVersion=2022-11-28).
+- Changes to Pipeline components need to be baked into the operator and built
+  rapidly.
+- Hot fixes need to be provided on a "roll forward" basis. OLM Operators do not
+  support "rollback" mechanisms today. Mean time to revert an offending change,
+  rebuild, and deploy needs to be measured in hours.
+- The version of the deployed operator needs to be configurable via ArgoCD.
+  This is should be doable by using `kustomize` to patch the `CatalogSource` and `ImageContentSourcePolicy`.

book/pipeline-service.md

@@ -1,28 +1,50 @@
 # Pipeline Service
 
-Pipeline Service provides a SaaS for pipelines. It leverages:
+Pipeline Service provides Tekton APIs and services to RHTAP.
+In the initial phase of RHTAP, Pipeline Service will be provided by a stock
+installation of the OpenShift Pipelines operator.
+This deployed version will be the a candidate build of the OpenShift Pipelines
+operator from a Red Hat build system.
 
-- OpenShift for the compute.
-- Tekton Pipelines, Chains, Pipelines as Code, Results for the core of the service.
-- OpenShift GitOps / Argo CD for managing the infrastructure.
+![Pipelines operator deployment](../diagrams/pipeline-service.drawio.svg)
 
-The following features are provided:
+## APIs and Services
 
-- Signing and attestation of `TaskRuns` with [Tekton Chains](https://tekton.dev/docs/chains/).
-- Archiving/Pruning of `PipelineRuns` and `TaskRuns` with [Tekton Results](https://tekton.dev/docs/results/) (not activated).
-- Integrations with GitHub, Gitlab, and Bitbucket with [Pipelines as Code](https://pipelinesascode.com) (PaC).
+Pipeline Service provides the following:
+
+- Tekton APIs directly through its custom resource definitions.
+- Container image signing and provenance attestations through Tekton Chains.
+- Archiving of `PipelineRuns`, `TaskRuns`, and associated logs through Tekton
+  Results.
+
+Pipeline Service also exposes the following ingress points:
+
+- Pipelines as Code controller: this is a `Route` that receives webhook events
+  from source code repositories.
+- Tekton Results API: this is an `Ingress` that serves Tekton Results data
+  over a RESTful API. Clients authenticate with the same `Bearer` token used to
+  authenticate Kubernetes requests.
+
+## Deployment Configuration
+
+The deployment of the OpenShift Pipelines operator will have the following
+notable configurations:
+
+- Tekton Triggers will be disabled entirely.
+- The pruner (provided by the Pipelines operator) will be disabled in favor of
+  pruning via Tekton Results.
+- Pipelines as Code will link the results of pipeline tasks to an appropriate
+  [Hybrid Application Console (HAC)](./hybrid-application-console.md) URL.
 
 ## Architecture
 
-### Component
+### Diagram
 
-The Pipeline Service component is deployed onto the cluster alongside other AppStudio components. The following forms the core architecture of Pipeline Service:
+Legend:
+* Blue: managed by Pipeline Service
+* Yellow: not managed by Pipeline Service
 
-1. _OpenShift Pipelines (OSP):_ It enables the creation and management of Pipelines for user [Applications](https://redhat-appstudio.github.io/book/ref/application-environment-api.html#application).
-2. _Pipelines as Code (PaC):_ An OpenShift Pipelines component that enables receiving webhook events from version control systems. The user would have to install a [GitHub App](https://pipelinesascode.com/docs/install/github_apps/) with the required permissions in order to forward events to the PaC endpoint exposed by the platform.
-3. _Spray Proxy:_ Forwards requests to all clusters in the environment once an event is received on PaC. It is a reverse proxy that broadcasts to multiple backends.
-4. _Tekton Results:_ Responsible for storing logs and pruning PipelineRuns. Utilises AWS RDS as the backend database for our production instance.
-5. _Tekton Chains:_ Provides PipelineRun attestation capabilities, ensuring that all PipelineRuns are verified and can be traced back to their source.
+![Architecture diagram](../diagrams/pipeline-service/architecture.jpg)
 
 ### appstudio-pipeline Service Account
 
@@ -35,4 +57,4 @@ The [CodeReadyToolchain](https://github.com/codeready-toolchain) platform (CRT)
 
 ## Repository
 
-The official repository for the Pipeline Service can be found at https://github.com/openshift-pipelines/pipeline-service. This repository contains the source code, configuration files, and documentation needed to deploy and consume the service.
+The official repository for the Pipeline Service can be found at https://github.com/openshift-pipelines/pipeline-service. This repository contains the source code, configuration files, and documentation needed to deploy and consume the service.