feat: gate-operator (#107)

* init * move sync loop to service reconciler * init service controller * refactor status collector * refactor status collector move engine utils to reconciler * move service controller to main * remove unused * add controller manager * add context * refactor * fix controller manager init * make console client once * refactor agent and engine * update socket * bump console client * refactor status * update socket * fix format error * fix some linter issues * update dep * refactor cmd package * fix build * minor changes * add lua unit tests * fix imports * minor enhancement * add restore controller * add velero crds * add backup controller * bump console client * add backup creation logs * add backup creation logs * copy full files * integrate agent changes * pipelines client * engine changes * add gateloop * fix * fix instantiation * gatecontrolloop as parallel * fix some stuff, type is always pending, api error? * needs namespace now, so has to be type job, or ns is added to api type * fix creation error + pretty print * missed adding to manager from old branch * restart policy onfailure per default * dont reconcile based on label/annotations changes, those are applied automatically * try fix for client init * create a second pipeline for testing * change name to name + id * pipeline3 * raw example * fix raw job parsing error * go mod tidy * add jobref to status field * refactor to track child job in status field jobref * actually always create the pipelinegate CR * add spec fields for status synced * add spec fields * actually lose ggraphql types * state needs to be nullable * all status fields need to be nullable * complete refactor of state handling * the time field is optional, should be becase you should be ablte to create the cr with kubectl * scribbles on state * debugg ln * make geenrate * make optional * fix the infinity jobs * fix yaml name * logging * refactor logging * rename field * fix job reconciliation error on resource version change * tests with cr * gateloop logging * clean up * clean up readme * move apis dir into api dir * adapt to new format * need by id for cache * pieplinegate reconciler * make some methods available outside of service pkg * add socket publisher * add gate controller * rm rebase leftovers * pipelinegate controllers rename * add capi * fix compilation * comment out scope, uses cluster api * use reconcile helper * fix version * use patch * rename package * fix rerun, and jobs not getting updated after connection/auth was lost for some time * clean up logging * rm comments * rm debug lines * convert fmt prints to logs * rm debug lines * update gate jobref at console after first reconcile * rm restore_controller.go * fix unit test * fix some lints * add pipelinegates crds to crd in charts * add permissions * fix * needs batch permissions, annotation for completeness sake * make test run * missing paths * Update pkg/controller/pipelinegates/reconciler.go Co-authored-by: Marcin Maciaszczyk <[email protected]> * Update controllers/pipelinegates/pipelinegate_controller.go Co-authored-by: Marcin Maciaszczyk <[email protected]> * move pipelinegate types over to v1alpha1 * create manifests for it * rbac for pipelines in deployments group * import pipelines from v1alpha1 * use ctrl runtime client instead of clientset * rm clientset usage * defer unnecessary * move synced state into status * deepcopy, too * role updated for group version * reset to pending in reconcile, needs to get resource from cluster for that * refactor controller with methods * synced state now in status * improve jobstatus handling * fix wrong condition * fix condition * use scope and remove debug lines * replace crd for pipelinegates in chart * update role for pipelinegates in new group * delete generated clientsets * move samples to config/samples * use cluster gate by id * dont need the client gen anymore * rm pipeilne query example * get rid of generic cache * create diableclientlimits in utils * sue utils version of disableclientlimits * no fmt here * rm velero crds * remove old pipelines types * after rm of velero crds # components is 23 * readd other controllers * delete notes * move pg controller over to other controllers * rn pipelinesv1alpha -> v1alhpa1 * rm test file * adapt dockerfile for re/moved packages * try fix chart linting action * bump chart version * fix lints * new line eof * fix some lints --------- Co-authored-by: Lukasz Zajaczkowski <[email protected]> Co-authored-by: Marcin Maciaszczyk <[email protected]>
pluralsh · Feb 9, 2024 · 79279f5 · 79279f5
1 parent 1fbbfc4
commit 79279f5
Show file tree

Hide file tree

Showing 36 changed files with 27,667 additions and 38 deletions.
diff --git a/.github/workflows/lint-and-test.yml b/.github/workflows/lint-and-test.yml
@@ -21,7 +21,7 @@ jobs:
 
       - name: Setup Chart Linting
         id: lint
-        uses: helm/chart-testing-action@v2.4.0
+        uses: helm/chart-testing-action@v2.6.1
 
       - name: List changed charts
         id: list-changed

diff --git a/.gitignore b/.gitignore
@@ -18,3 +18,5 @@ bin/
 
 # Dependency directories (remove the comment below to include it)
 # vendor/
+
+notes.md
diff --git a/api/v1alpha1/doc.go b/api/v1alpha1/doc.go
@@ -0,0 +1,3 @@
+// +groupName=deployments.plural.sh
+
+package v1alpha1
diff --git a/api/v1alpha1/pipelinegate_types.go b/api/v1alpha1/pipelinegate_types.go
@@ -0,0 +1,104 @@
+/*
+Copyright 2021.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package v1alpha1
+
+import (
+	console "github.com/pluralsh/console-client-go"
+	batchv1 "k8s.io/api/batch/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+)
+
+// +kubebuilder:validation:Enum=PENDING;OPEN;CLOSED
+// GateState represents the state of a gate, reused from console client
+type GateState console.GateState
+
+// +kubebuilder:validation:Enum=APPROVAL;WINDOW;JOB
+// GateType represents the type of a gate, reused from console client
+type GateType console.GateType
+
+//+genclient
+//+kubebuilder:object:root=true
+//+kubebuilder:subresource:status
+
+// PipelineGate represents a gate blocking promotion along a release pipeline
+type PipelineGate struct {
+	metav1.TypeMeta   `json:",inline"`
+	metav1.ObjectMeta `json:"metadata,omitempty"`
+
+	Spec   PipelineGateSpec   `json:"spec,omitempty"`
+	Status PipelineGateStatus `json:"status,omitempty"`
+}
+
+// PipelineGateStatus defines the observed state of the PipelineGate
+type PipelineGateStatus struct {
+	State          *GateState              `json:"state,omitempty"`
+	SyncedState    GateState               `json:"syncedState"`
+	LastSyncedAt   metav1.Time             `json:"lastSyncedAt,omitempty"`
+	LastReported   *GateState              `json:"lastReported,omitempty"`
+	LastReportedAt *metav1.Time            `json:"lastReportedAt,omitempty"`
+	JobRef         *console.NamespacedName `json:"jobRef,omitempty"`
+}
+
+// PipelineGateSpec defines the detailed gate specifications
+type PipelineGateSpec struct {
+	ID       string    `json:"id"`
+	Name     string    `json:"name"`
+	Type     GateType  `json:"type"`
+	GateSpec *GateSpec `json:"gateSpec,omitempty"`
+}
+
+// GateSpec defines the detailed gate specifications
+type GateSpec struct {
+	// resuse JobSpec type from the kubernetes api
+	JobSpec *batchv1.JobSpec `json:"job"`
+}
+
+// +kubebuilder:object:root=true
+// PipelineGateList contains a list of PipelineGate
+type PipelineGateList struct {
+	metav1.TypeMeta `json:",inline"`
+	metav1.ListMeta `json:"metadata,omitempty"`
+	Items           []PipelineGate `json:"items"`
+}
+
+func init() {
+	SchemeBuilder.Register(&PipelineGate{}, &PipelineGateList{})
+}
+
+func (pgs *PipelineGateStatus) IsInitialized() bool {
+	return pgs.State != nil
+}
+
+func (pgs *PipelineGateStatus) IsPending() bool {
+	return pgs.State != nil && *pgs.State == GateState(console.GateStatePending)
+}
+
+func (pgs *PipelineGateStatus) IsOpen() bool {
+	return pgs.State != nil && *pgs.State == GateState(console.GateStateOpen)
+}
+
+func (pgs *PipelineGateStatus) IsClosed() bool {
+	return pgs.State != nil && *pgs.State == GateState(console.GateStateClosed)
+}
+
+func (pgs *PipelineGateStatus) HasJobRef() bool {
+	return !(pgs.JobRef == nil || *pgs.JobRef == console.NamespacedName{})
+}
+
+func (pgs *PipelineGateStatus) HasNotReported() bool {
+	return pgs.LastReported == nil || (pgs.LastReportedAt != nil && pgs.LastReportedAt.Before(&pgs.LastSyncedAt))
+}
diff --git a/api/v1alpha1/register.go b/api/v1alpha1/register.go
@@ -0,0 +1,12 @@
+package v1alpha1
+
+import (
+	"k8s.io/apimachinery/pkg/runtime/schema"
+)
+
+// SchemeGroupVersion is group version used to register these objects.
+var SchemeGroupVersion = GroupVersion
+
+func Resource(resource string) schema.GroupResource {
+	return SchemeGroupVersion.WithResource(resource).GroupResource()
+}
diff --git a/api/v1alpha1/zz_generated.deepcopy.go b/api/v1alpha1/zz_generated.deepcopy.go
diff --git a/charts/deployment-operator/Chart.yaml b/charts/deployment-operator/Chart.yaml
@@ -2,8 +2,8 @@ apiVersion: v2
 name: deployment-operator
 description: creates a new instance of the plural deployment operator
 type: application
-version: 0.4.6
-appVersion: "0.4.5"
+version: 0.4.7
+appVersion: "0.4.7"
 maintainers:
 - name: Plural
   url: https://www.plural.sh