[opentelemetry] add plugin config to enable or disable metri… (#320)

* [opentelemetry-operator] add plugin config to enable or disable metric and logs collector

* [opentelemetry-operator] rename plugin, update documentation

* [opentelemetry] add trailing line

* [opentelemetry] add -operator to helm-chart name

* Apply suggestions from code review

Co-authored-by: Richard Tief <[email protected]>
Co-authored-by: Tommy Sauer <[email protected]>

* [opentelemetry] incorporate changes from review

* [opentelemetry] update codeowners to rm -operator

---------

Co-authored-by: Richard Tief <[email protected]>
Co-authored-by: Tommy Sauer <[email protected]>

.github/CODEOWNERS

@@ -28,7 +28,7 @@ ct.yaml                   @cloudoperators/greenhouse-backend
 /kube-monitoring/         @cloudoperators/greenhouse-observability
 /kubeconfig-generator/    @Nuckal777 @ivogoman @uwe-mayer
 /logshipper/              @cloudoperators/greenhouse-observability @cloudoperators/greenhouse-backend
-/opentelemetry-operator   @cloudoperators/greenhouse-observability
+/opentelemetry/           @cloudoperators/greenhouse-observability
 /plutono                  @cloudoperators/greenhouse-observability 
 /service-proxy/           @cloudoperators/greenhouse-backend @databus23 
 /teams2slack/             @cloudoperators/greenhouse-backend @voigts 

docs/catalog.md

@@ -16,4 +16,4 @@ This following table provides an overview of the currently available Plugins in
 | ingress-nginx|Ingress NGINX controller|1.1.0|
 | kube-monitoring|Native deployment and management of Prometheus along with Kubernetes cluster monitoring components.|1.3.4|
 | teams2slack|Manage Slack handles and channels based on Greenhouse teams and their members|1.1.0|
-| opentelemetry-operator|This Plugin is intended for ingesting, generating, collecting, and exporting telemetry data (metrics, logs, and traces)|0.1.0|
+| opentelemetry|This Plugin is intended for ingesting, generating, collecting, and exporting telemetry data (metrics, logs, and traces)|0.2.7|

opentelemetry/README.md

@@ -0,0 +1,50 @@
+---
+title: OpenTelemetry
+---
+
+This Plugin is intended for ingesting, generating, collecting, and exporting telemetry data (metrics, logs, and traces). 
+
+Components included in this Plugin:
+
+- [Collector](https://github.com/open-telemetry/opentelemetry-collector)
+- [Filelog Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver)
+- [OpenSearch Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/opensearchexporter)
+- ServiceMonitor to ingest the metrics of the OpenTelemetry Collector in Prometheus 
+
+The Plugin will deploy the OpenTelemetry Operator which works as a manager for the collectors and auto-instrumentation of the workload. By default the Plugin will also 
+1. Collect logs via the Filelog Receiver, process and export them to OpenSearch
+    - You can disable the collecting of logs by setting `open_telemetry.LogCollector.enabled` to `false`. 
+2. Collect and expose metrics using a Prometheus interface which defaults to port 8888
+    - You can disable the collecting of metrics by setting `open_telemetry.MetricsCollector.enabled` to `false`. 
+
+Contributors are welcome to provide additional collector configurations. 
+
+# Owner
+
+1. Timo Johner (@timojohlo) 
+2. Olaf Heydorn (@kuckkuck) 
+3. Tommy Sauer (@viennaa) 
+
+### Parameters
+| Name         | Description          | Type           | required           |
+| ------------ | -------------------- |---------------- | ------------------ | 
+`openTelemetry.logsCollector.enabled`    | Activates the standard configuration for logs | bool | `false`
+`openTelemetry.metricsCollector.enabled` | Activates the standard configuration for metrics | bool | `false`
+`openTelemetry.openSearchLogs.username` | Username for OpenSearch endpoint | secret | `false` |
+`openTelemetry.openSearchLogs.password` | Password for OpenSearch endpoint | secret | `false` | 
+`openTelemetry.openSearchLogs.endpoint` | Endpoint URL for OpenSearch      | secret | `false` | 
+`openTelemetry.region`                   | Region label for logging         | string | `false` |
+`openTelemetry.cluster`                  | Cluster label for logging        | string | `false` |
+`openTelemetry.prometheus`               | Label for Prometheus Service Monitoring | string | `false` | 
+`openTelemetry.podMonitor.enabled`       | Activates the pod-monitoring for the LogsCollector | bool | `false` | 
+`opentelemetry-operator.admissionWbhooks.certManager.enabled` | Activate to use the CertManager for generating self-signed certificates | bool | `false` | 
+`opentelemetry-operator.admissionWebhooks.autoGenerateCert.enabled` | Activate to use Helm to create self-signed certificates | bool | `false` | 
+`opentelemetry-operator.admissionWebhooks.autoGenerateCert.recreate` | Activate to recreate the cert after a defined period (certPeriodDays default is 365) | bool | `false` | 
+`opentelemetry-operator.kubeRBACProxy.enabled` | Activate to enable Kube-RBAC-Proxy for OpenTelemetry | bool | `false` | 
+`opentelemetry-operator.manager.prometheusRule.defaultRules.enabled` | Activate to enable default rules for monitoring the OpenTelemetry Manager | bool | `false` | 
+`opentelemetry-operator.manager.prometheusRule.enabled` | Activate to enable rules for monitoring the OpenTelemetry Manager | bool | `false` | 
+`opentelemetry-operator.manager.serviceMonitor.enabled` | Activate to enable the Service Monitor to ingest the metrics of the OpenTelemetry Collector in Prometheus  | bool | `false` | 
+
+### Examples
+
+TBD

opentelemetry-operator/chart/Chart.yaml → opentelemetry/chart/Chart.yaml

@@ -4,7 +4,7 @@
 apiVersion: v2
 appVersion: v0.104.0
 name: opentelemetry-operator
-version: 0.2.6
+version: 0.2.7
 description: OpenTelemetry Operator Helm chart for Kubernetes
 icon: https://raw.githubusercontent.com/cncf/artwork/a718fa97fffec1b9fd14147682e9e3ac0c8817cb/projects/opentelemetry/icon/color/opentelemetry-icon-color.png
 type: application

opentelemetry-operator/chart/templates/logs-collector.yaml → opentelemetry/chart/templates/logs-collector.yaml

@@ -3,6 +3,7 @@ SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse c
 SPDX-License-Identifier: Apache-2.0
 */}}
 
+{{- if .Values.openTelemetry.logsCollector.enabled }}
 apiVersion: opentelemetry.io/v1beta1
 kind: OpenTelemetryCollector
 metadata:
@@ -36,10 +37,10 @@ spec:
           name: otel-basic-auth
           key: password          
     - name: cluster
-      value: "{{ .Values.open_telemetry.cluster }}"
+      value: "{{ .Values.openTelemetry.cluster }}"
     - name: region
-      value: "{{ .Values.open_telemetry.region }}"
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+      value: "{{ .Values.openTelemetry.region }}"
+{{- if .Values.openTelemetry.podMonitor.enabled }}
   ports:
     - name: prometheus
       port: 9999
@@ -126,7 +127,7 @@ spec:
             from: attributes.uid
             to: resource["k8s.pod.uid"]
 
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+{{- if .Values.openTelemetry.podMonitor.enabled }}
       prometheus/internal:
         config:
           scrape_configs:
@@ -142,7 +143,7 @@ spec:
         timeout: 10s
         send_batch_size : 400
 
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+{{- if .Values.openTelemetry.podMonitor.enabled }}
       attributes/prometheus:
         actions:
           - action: insert
@@ -295,7 +296,7 @@ spec:
         http:
           auth:
             authenticator: basicauth
-          endpoint: {{ index .Values "open_telemetry" "opensearch_logs" "endpoint" }}
+          endpoint: {{ .Values.openTelemetry.openSearchLogs.endpoint }}
         logs_index: ${username}-datastream
 
       prometheus:
@@ -318,7 +319,7 @@ spec:
     service:
       extensions:
         - basicauth
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+{{- if .Values.openTelemetry.podMonitor.enabled }}
       telemetry:
         metrics:
           address: 127.0.0.1:8888
@@ -337,9 +338,10 @@ spec:
           receivers: [journald]
           processors: [attributes/cluster,transform/journal,batch]
           exporters: [opensearch/logs]
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+{{- if .Values.openTelemetry.podMonitor.enabled }}
         metrics/prometheus:
           receivers: [prometheus/internal]
           processors: [attributes/prometheus]
           exporters: [prometheus]
 {{- end }}
+{{- end }}

opentelemetry-operator/chart/templates/metrics-collector.yaml → opentelemetry/chart/templates/metrics-collector.yaml

@@ -3,6 +3,7 @@ SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse c
 SPDX-License-Identifier: Apache-2.0
 */}}
 
+{{- if .Values.openTelemetry.metricsCollector.enabled }}
 apiVersion: opentelemetry.io/v1beta1
 kind: OpenTelemetryCollector
 metadata:
@@ -46,3 +47,4 @@ spec:
         metrics:
           level: detailed
           address: 0.0.0.0:8888
+{{- end }}

opentelemetry-operator/chart/templates/secret.yaml → opentelemetry/chart/templates/secret.yaml

@@ -8,5 +8,5 @@ kind: Secret
 metadata:
   name: otel-basic-auth
 data:
-   password: {{ index .Values "open_telemetry" "opensearch_logs" "password" | b64enc }}
-   username: {{ index .Values "open_telemetry" "opensearch_logs" "username" | b64enc }}
+   password: {{ .Values.openTelemetry.openSearchLogs.password | b64enc }}
+   username: {{ .Values.openTelemetry.openSearchLogs.username | b64enc }}

opentelemetry-operator/chart/templates/smon-filelog.yaml → opentelemetry/chart/templates/smon-filelog.yaml

@@ -2,14 +2,14 @@
 SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse contributors
 SPDX-License-Identifier: Apache-2.0
 */}}
-{{- if index .Values "open_telemetry" "podMonitor" "enabled" }}
+{{- if .Values.openTelemetry.podMonitor.enabled }}
 apiVersion: monitoring.coreos.com/v1
 kind: PodMonitor
 metadata:
   annotations:
   name: opentelemetry-collector-logs
   labels:
-    prometheus: "{{ .Values.open_telemetry.prometheus }}"
+    prometheus: "{{ .Values.openTelemetry.prometheus }}"
 spec:
   podMetricsEndpoints:
     - interval: 60s

opentelemetry-operator/plugindefinition.yaml → opentelemetry/plugindefinition.yaml

@@ -4,77 +4,88 @@
 apiVersion: greenhouse.sap/v1alpha1
 kind: PluginDefinition
 metadata:
-    name: opentelemetry-operator
+    name: opentelemetry
 spec:
-    version: 0.2.6
-    description: opentelemetry-operator
+    version: 0.2.7
+    displayName: OpenTelemetry
+    description: Observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, and logs.
     icon: https://raw.githubusercontent.com/cloudoperators/greenhouse-extensions/main/opentelemetry-operator/logo.png
     helmChart:
         name: opentelemetry-operator
         repository: oci://ghcr.io/cloudoperators/greenhouse-extensions/charts
-        version: 0.2.6
+        version: 0.2.7
     options:
+        - default: true
+          description: Activates the standard configuration for logs
+          name: openTelemetry.logsCollector.enabled
+          required: false
+          type: bool
+        - default: true
+          description: Activates the standard configuration for metrics 
+          name: openTelemetry.metricsCollector.enabled
+          required: false
+          type: bool
         - description: Username for OpenSearch endpoint
-          name: open_telemetry.opensearch_logs.username
+          name: openTelemetry.openSearchLogs.username
           required: false
           type: secret
         - description: Password for OpenSearch endpoint
-          name: open_telemetry.opensearch_logs.password
+          name: openTelemetry.openSearchLogs.password
           required: false
           type: secret
         - description: Endpoint URL for OpenSearch
-          name: open_telemetry.opensearch_logs.endpoint
+          name: openTelemetry.openSearchLogs.endpoint
           required: false
           type: string
-        - description: Region
-          name: open_telemetry.region
+        - description: Region label for logging
+          name: openTelemetry.region
           required: false
           type: string
-        - description: Cluster
-          name: open_telemetry.cluster
+        - description: Cluster label for logging
+          name: openTelemetry.cluster
           required: false
           type: string
         - description: Label for Prometheus Service Monitoring
-          name: open_telemetry.prometheus
+          name: openTelemetry.prometheus
           required: false
           type: string
         - default: false
-          description: Check to enable the Pod Monitor
-          name: open_telemetry.podMonitor.enabled
+          description: Activates the pod-monitoring for the Logs Collector
+          name: openTelemetry.podMonitor.enabled
           required: false
           type: bool
         - default: false
-          description: Check to use certManager for generating self-signed certificates
+          description: Activate to use the CertManager for generating self-signed certificates
           name: opentelemetry-operator.admissionWebhooks.certManager.enabled
           required: false
           type: bool
         - default: true
-          description: Check to use Helm to create self-signed certificates
+          description: Activate to use Helm to create self-signed certificates
           name: opentelemetry-operator.admissionWebhooks.autoGenerateCert.enabled
           required: false
           type: bool
         - default: false
-          description: Recreate the cert after a defined period (certPeriodDays default is 365)
+          description: Activate to recreate the cert after a defined period (certPeriodDays default is 365)
           name: opentelemetry-operator.admissionWebhooks.autoGenerateCert.recreate
           required: false
           type: bool
         - default: false
-          description: Check to enable kube-rbac-proxy for OpenTelemetry
+          description: Activate to enable Kube-RBAC-Proxy for OpenTelemetry
           name: opentelemetry-operator.kubeRBACProxy.enabled
           required: false
           type: bool
         - default: false
-          description: Check to enable default rules for monitoring the manager
+          description: Activate to enable default rules for monitoring the OpenTelemetry Manager
           name: opentelemetry-operator.manager.prometheusRule.defaultRules.enabled
           required: false
           type: bool
         - default: false
-          description: Check to enable rules for monitoring the manager
+          description: Activate to enable rules for monitoring the OpenTelemetry Manager
           name: opentelemetry-operator.manager.prometheusRule.enabled
           required: false
           type: bool
         - default: false
-          description: Check to enable the Service Monitor
+          description: Activate to enable the Service Monitor to ingest the metrics of the OpenTelemetry Collector in Prometheus
           name: opentelemetry-operator.manager.serviceMonitor.enabled
           required: false
           type: bool