From 3cc70ec32fac246716dbf2aa944894c5df4438ea Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 11:52:31 +0300 Subject: [PATCH 01/20] Update chart/app version and README Signed-off-by: JeffMboya --- charts/magistrala/Chart.yaml | 4 +- charts/magistrala/README.md | 243 +++++------------------------------ 2 files changed, 35 insertions(+), 212 deletions(-) diff --git a/charts/magistrala/Chart.yaml b/charts/magistrala/Chart.yaml index e9ceb984..c644d474 100644 --- a/charts/magistrala/Chart.yaml +++ b/charts/magistrala/Chart.yaml @@ -6,8 +6,8 @@ name: magistrala description: Magistrala IoT Platform icon: https://avatars1.githubusercontent.com/u/13207490 type: application -version: 1.0.4 # Incremented chart version if the chart is updated -appVersion: "0.12.2" # Update application version if the app is updated +version: 1.0.6 # Incremented chart version if the chart is updated +appVersion: "0.14.0" # Update application version if the app is updated home: https://abstractmachines.fr/magistrala.html sources: - https://hub.docker.com/u/magistrala diff --git a/charts/magistrala/README.md b/charts/magistrala/README.md index 947cff86..12022dad 100644 --- a/charts/magistrala/README.md +++ b/charts/magistrala/README.md @@ -1,238 +1,61 @@ -# Magistrala Helm Chart +## Usage -Helm Chart for the Magistrala IoT Platform. +[Helm](https://helm.sh) must be installed to use Magistrala charts. Please refer to Helm's [documentation](https://helm.sh/docs) to get started if you haven't already. -## Prerequisites +### Step 1: Adding the Magistrala DevOps Repo -- Helm v3 -- Stable & Bitnami Helm repos installed - ``` - helm repo add stable https://charts.helm.sh/stable - helm repo add bitnami https://charts.bitnami.com/bitnami - helm repo add jaegertracing https://jaegertracing.github.io/helm-charts - ``` -- Nginx Ingress Controller -- If using the mTLS setup: - - Certificate/Key installed as**TLS secret** - - Intermediate certificate installed as**Generic secret** - -## Adding the Helm Repository - -The Helm charts are published via GitHub Pages. To add the repository to your Helm configuration, run the following command: +Once Helm has been set up correctly, add the Magistrala DevOps repository by running the following command: ```bash -helm repo add devops-charts https://absmach.github.io/devops/ +helm repo add https://absmach.github.io/devops/ ``` -Update your local Helm repository cache to fetch the latest charts: +`` is the alias or label that you assign to the repository you are adding. It can be any name of your choice, and it will act as a reference for the repository in your subsequent Helm commands. + +For example: ```bash -helm repo update +helm repo add magistrala-devops https://absmach.github.io/devops/ ``` -## Installing the Chart +### Step 2: Installing the Chart -Once the repository is added, you can install the chart using Helm. Replace `` with your desired release name: +To install the Magistrala chart, run the following command: ```bash -helm install devops-charts/magistrala +helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 ``` -This command will install the `magistrala` chart from the GitHub Pages-hosted Helm repository. +* `my-magistrala` is the release name you choose for the installation. You can modify this to suit your needs. +* `magistrala-devops/magistrala` specifies the chart name from the added repository. +* `--version 1.0.6` ensures you install the specific version of the chart. + +* To override values in the chart, use either the `--values` flag and pass in a file or use the `--set` flag and pass configuration from the command line. -## Upgrading the Chart +### Step 3: Updating the Repository -To upgrade the chart with a new version or updated configuration, use the following command: +If you had already added this repo earlier, run the following command to retrieve the latest versions of the packages: ```bash -helm upgrade devops-charts/magistrala +helm repo update ``` -This ensures that your deployment uses the latest version of the chart while retaining any custom configurations. +### Step 4: Searching for Charts -## Uninstalling the Chart +You can search the repository for a keyword in charts using the following command: -To uninstall the chart and release, run: +```bash +helm search repo [keyword] [flags] +``` + +### Step 5: Uninstalling the Chart + +To uninstall the chart when you no longer need it, run the following command: ```bash -helm uninstall +helm uninstall my-magistrala [flags] ``` -## Configuration - -The following table lists the configurable parameters and their default values. - -| Parameter | Description | Default | -| ----------------------------------------- | -------------------------------------------------------------------------- | -------------------- | -| defaults.logLevel | Default log level | info | -| defaults.image.pullPolicy | Default Image Pull Policy | IfNotPresent | -| defaults.image.rootRepository | Default Image root repository for magistrala service | magistrala | -| defaults.image.tag | Default Image Tag | latest | -| defaults.replicaCount | Replicas of MQTT adapter, NATS, Things, Envoy, and Auth | 3 | -| defaults.natsPort | NATS port | 4222 | -| defaults.jaegerCollectorPort | Jaeger collector port | 4318 | -| defaults.jaegerTraceRatio | Jaeger trace ratio (percentage of traces to be sampled) | 10 | -| defaults.sendTelemetry | Enable or disable sending telemetry | true | -| defaults.eventStreamURL | Event stream URL for NATS | magistrala-nats:4222 | -| ingress.enabled | Should the Nginx Ingress be created | true | -| ingress.annotations | Annotations for the Nginx Ingress | {} | -| ingress.labels | Labels for the Nginx Ingress | {} | -| ingress.hostname | Hostname for the Nginx Ingress | | -| ingress.tls.hostname | TLS hostname for the Nginx Ingress | | -| ingress.tls.scret | TLS sceret for the Nginx Ingress | | -| nginxInternal.image.pullPolicy | internal nginx image pull policy | IfNotPresent | -| nginxInternal.image.repository | inernal nginx image repository | nginx | -| nginxInternal.image.tag | inernal nginx image tag | 1.19.1-alpine | -| nginxInternal.mtls.tls | TLS secret which contains the server cert/key | | -| nginxInternal.mtls.intermediateCrt | Generic secret which contains the intermediate cert used to verify clients | | -| envoy.image.pullPolicy | Envoy Image Pull Policy for Envoy | IfNotPresent | -| envoy.image.repository | Envoy Image repository for Envoy | envoyproxy/envoy | -| envoy.image.tag | Envoy Image Tag for Envoy | v1.31-latest | -| jaeger.fullnameOverride | Jaeger fullname override | magistrala-jaeger | -| jaeger.provisionDataStore.cassandra | Provision Jaeger with Cassandra data store | true | -| jaeger.agent.enabled | Enable Jaeger agent | false | -| jaeger.allInOne.enabled | Enable Jaeger all-in-one mode | false | -| jaeger.storage.type | Storage type for Jaeger | memory | -| jaeger.collector.service.otlp.grpc.port | OTLP gRPC port for Jaeger collector | 4317 | -| jaeger.collector.service.otlp.http.port | OTLP HTTP port for Jaeger collector | 4318 | -| nats.config.cluster.enabled | Enable NATS clustering | false | -| nats.config.cluster.replicas | Number of replicas in NATS cluster | 3 | -| nats.config.jetstream.enabled | Enable JetStream for NATS | true | -| nats.config.jetstream.fileStore.enabled | Enable file storage for JetStream | true | -| nats.config.jetstream.memoryStore.enabled | Enable memory storage for JetStream | true | -| nats.config.jetstream.memoryStore.maxSize | Maximum size for JetStream memory storage | 2Gi | -| adapter_coap.image.pullSecrets | CoAP adapter image pull secrets | | -| adapter_coap.image.repository | CoAP adapter image repository | | -| adapter_coap.image.tag | CoAP adapter image tag | | -| adapter_coap.image.pullPolicy | CoAP adapter image pull policy | | -| adapter_coap.port | CoAP adapter UDP port | 5683 | -| adapter_http.image.pullSecrets | HTTP adapter image pull secrets | | -| adapter_http.image.repository | HTTP adapter image repository | | -| adapter_http.image.tag | HTTP adapter image tag | | -| adapter_http.image.pullPolicy | HTTP adapter image pull policy | | -| adapter_http.httpPort | HTTP adapter port | 8008 | -| mqtt.enabled | Enable MQTT adapter | true | -| mqtt.securityContext.runAsUser | Run MQTT adapter as a specific user | 10000 | -| mqtt.securityContext.runAsGroup | Run MQTT adapter as a specific group | 10000 | -| mqtt.securityContext.fsGroup | Filesystem group for MQTT adapter | 10000 | -| mqtt.adapter.image.pullSecrets | MQTT adapter image pull secrets | | -| mqtt.adapter.image.repository | MQTT adapter image repository | | -| mqtt.adapter.image.tag | MQTT adapter image tag | | -| mqtt.adapter.image.pullPolicy | MQTT adapter image pull policy | | -| mqtt.adapter.mqttPort | MQTT adapter MQTT port | 1884 | -| mqtt.adapter.wsPort | MQTT adapter WebSocket port | 8081 | -| mqtt.adapter.logLevel | Log level for MQTT adapter | debug | -| mqtt.broker.mqttPort | MQTT broker MQTT port | 1883 | -| mqtt.broker.wsPort | MQTT broker WebSocket port | 8080 | -| mqtt.broker.logLevel | Log level for MQTT broker | info | -| mqtt.broker.persistentVolume.size | Persistent volume size for MQTT broker | 5Gi | -| mqtt.redisESPort | MQTT adapter Redis Event Store port | 6379 | -| mqtt.redisCachePort | MQTT adapter Redis Cache port | 6379 | -| spicedb.image.repository | Docker Image repository for SpiceDB | authzed/spicedb | -| spicedb.image.tag | Docker Image Tag for SpiceDB | latest | -| spicedb.grpc.presharedKey | Pre-shared key for SpiceDB gRPC | helloworld | -| spicedb.grpc.port | SpiceDB gRPC port | 50051 | -| spicedb.datastore.engine | Datastore engine for SpiceDB | postgres | -| spicedb.dispatch.port | Dispatch port for SpiceDB | 50053 | -| spicedb.dispatch.enabled | Enable Dispatch for SpiceDB | false | -| spicedb.http.enabled | Enable HTTP for SpiceDB | false | -| spicedb.http.port | HTTP port for SpiceDB | 8443 | -| spicedb.metrics.enabled | Enable metrics for SpiceDB | true | -| spicedb.metrics.port | Metrics port for SpiceDB | 9090 | -| postgresqlspicedb.enabled | Enable PostgreSQL for SpiceDB | true | -| postgresqlspicedb.host | Host for PostgreSQL SpiceDB | postgresql-spicedb | -| postgresqlspicedb.port | PostgreSQL port for SpiceDB | 5432 | -| postgresqlspicedb.database | Database name for SpiceDB | spicedb | -| postgresqlspicedb.username | Username for PostgreSQL SpiceDB | magistrala | -| postgresqlspicedb.password | Password for PostgreSQL SpiceDB | magistrala | -| auth.httpPort | HTTP port for Auth service | 8189 | -| auth.grpcPort | gRPC port for Auth service | 8181 | -| auth.secret | Secret key for Auth service | supersecret | -| auth.adminEmail | Admin email for Auth service | admin@example.com | -| auth.adminPassword | Admin password for Auth service | 12345678 | -| auth.accessTokenDuration | Access token duration for Auth service | 1h | -| auth.refreshTokenDuration | Refresh token duration for Auth service | 24h | -| auth.invitationDuration | Invitation duration for Auth service | 168h | -| postgresqlauth.enabled | Enable PostgreSQL for Auth service | true | -| postgresqlauth.host | Host for PostgreSQL Auth service | postgresql-auth | -| postgresqlauth.port | PostgreSQL port for Auth service | 5432 | -| postgresqlauth.database | Database name for Auth service | auth | -| postgresqlauth.username | Username for PostgreSQL Auth service | magistrala | -| postgresqlauth.password | Password for PostgreSQL Auth service | magistrala | -| users.dbPort | Users service DB port | 5432 | -| users.httpPort | Users service HTTP port | 9002 | -| users.grpcPort | Users service gRPC port | 7001 | -| things.dbPort | Things service DB port | 5432 | -| things.httpPort | Things service HTTP port | 9000 | -| things.authGrpcPort | Things service Auth gRPC port | 7000 | -| things.authHttpPort | Things service Auth HTTP port | 9001 | -| things.redisESPort | Things service Redis Event Store port | 6379 | -| things.redisCachePort | Things service Redis Auth Cache port | 6379 | -| adapter_http.httpPort | HTTP adapter port | 8008 | -| mqtt.adapter.mqttPort | MQTT adapter port | 1884 | -| mqtt.adapter.wsPort | MQTT adapter WS port | 8081 | -| mqtt.broker.mqttPort | MQTT adapter broker port | 1883 | -| mqtt.broker.wsPort | MQTT adapter broker WS port | 8080 | -| mqtt.broker.persistentVolume.size | MQTT adapter broker data Persistent Volume size | 5Gi | -| mqtt.redisESPort | MQTT adapter Event Store port | 6379 | -| mqtt.redisCachePort | MQTT adapter Redis Auth Cache port | 6379 | -| adapter_coap.udpPort | CoAP adapter UDP port | 5683 | -| ui.port | UI port | 3000 | -| bootstrap.enabled | Enable bootstrap service | false | -| bootstrap.dbPort | Bootstrap service DB port | 5432 | -| bootstrap.httpPort | Bootstrap service HTTP port | 9013 | -| bootstrap.redisESPort | Bootstrap service Redis Event Store port | 6379 | -| influxdb.enabled | Enable InfluxDB reader & writer | false | -| influxdb.dbPort | InfluxDB port | 8086 | -| influxdb.writer.httpPort | InfluxDB writer HTTP port | 9006 | -| influxdb.reader.httpPort | InfluxDB reader HTTP port | 9005 | -| influxdb.backup.enabled | Enable InfluxDB backup | false | -| influxdb.backup.cronjob.schedule | Crontab style time schedule for backup execution | "0 2 \* \* \*" | -| adapter_opcua.enabled | Enable OPC-UA adapter | false | -| adapter_opcua.httpPort | OPC-UA adapter HTTP port | 8188 | -| adapter_opcua.redisRouteMapPort | OPC-UA adapter Redis Auth Cache port | 6379 | -| adapter_lora.enabled | Enable LoRa adapter | false | -| adapter_lora.httpPort | LoRa adapter HTTP port | 9017 | -| adapter_lora.redisRouteMapPort | LoRa adapter Redis Auth Cache port | 6379 | -| twins.enabled | Enable twins service | false | -| twins.dbPort | Twins service DB port | 27017 | -| twins.httpPort | Twins service HTTP port | 9018 | -| twins.redisCachePort | Twins service Redis Cache port | 6379 | -| certs.enabled | Enable certs service | false | -| notifier_smtp.enabled | Enable SMTP notifier | false | -| notifier_smtp.emailHost | SMTP host | false | -| notifier_smtp.smtpPort | SMTP port | false | -| notifier_smtp.fromName | SMTP notifier `from` name | false | -| notifier_smtp.fromEmail | SMTP `from` email address | false | -| notifier_smtp.username | SMTP username | false | -| notifier_smtp.password | SMTP password | false | -| notifier_smtp.secret | SMTP secret | false | -| notifier_smtp.httpPort | SMTP notifier HTTP port | false | -| loki_stack.enabled | Enable Loki_Stack | true | - -All Magistrala services (both core and add-ons) can have their `logLevel`, `image.pullPolicy`, `image.repository` and `image.tag` overridden. - -Magistrala Core is a minimalistic set of required Magistrala services. They are all installed by default: - -- auth -- users -- things -- adapter_http -- adapter_mqtt -- adapter_coap -- ui - -Magistrala Add-ons are optional services that are disabled by default. Find in Configuration table parameters for enabling them, i.e. to enable influxdb reader & writer you should run `helm install` with `--set influxdb=true`. -List of add-ons services in charts: - -- bootstrap -- influxdb.writer -- influxdb.reader -- adapter_opcua -- adapter_lora -- twins -- notifier_smtp - -By default scale of MQTT adapter, Things, Envoy, Auth and NATS will be set to 3. It's recommended that you set this values to number of your nodes in Kubernetes cluster, i.e. `--set defaults.replicaCount=3 --set nats.replicaCount=3` - -**Note:** make sure you run `helm install` with `--dependency-update` flag! +Make sure to replace `my-magistrala` with your actual release name if you changed it during installation. The command removes all of the resources associated with the last release of the chart as well as the release history, freeing it up for future use. + +--- \ No newline at end of file From ba0d0a92abaa1cde5c6a1c93bec79232f1050bed Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 12:04:10 +0300 Subject: [PATCH 02/20] Add dependencies to lint-test.yaml Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index de9ae9c0..24b5dcb3 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -19,6 +19,14 @@ jobs: with: version: v3.15.4 + - name: Add Helm dependencies + run: | + helm repo add nats https://nats-io.github.io/k8s/helm/charts/ + helm repo add jaegertracing https://jaegertracing.github.io/helm-charts + helm repo add bitnami https://charts.bitnami.com/bitnami + helm repo add hashicorp https://helm.releases.hashicorp.com + helm repo update + - uses: actions/setup-python@v5 with: python-version: "3.x" From 49aac53163f623b9f95fd415bd66d7312813fd25 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 12:22:23 +0300 Subject: [PATCH 03/20] Fix CI to lint magistrala chart Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 24b5dcb3..e68c2b73 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -45,7 +45,8 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' - run: ct lint --target-branch ${{ github.event.repository.default_branch }} + run: | + ct lint --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -53,4 +54,5 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' - run: ct install --target-branch ${{ github.event.repository.default_branch }} + run: | + ct install --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala From 125ccce38f4df67218570f040e98dc5cb8504ba4 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 12:29:09 +0300 Subject: [PATCH 04/20] remove extra spaces Signed-off-by: JeffMboya --- charts/magistrala/values.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/charts/magistrala/values.yaml b/charts/magistrala/values.yaml index 98102167..ba66439e 100644 --- a/charts/magistrala/values.yaml +++ b/charts/magistrala/values.yaml @@ -526,7 +526,7 @@ timescaledb: # sendTelemetry: true # logLevel: "info" enabled: true - http: { port: 9011 } + http: {port: 9011} # nodeSelector: {} # affinity: {} # tolerations: {} @@ -543,7 +543,7 @@ timescaledb: # affinity: {} # tolerations: {} enabled: true - http: { port: 9012 } + http: {port: 9012} ## Configurations of Bitnami postgres global: postgresql: From c229ad3e6631e3962cc18c3c02979adde4176b76 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 12:41:03 +0300 Subject: [PATCH 05/20] update trigger from main to master Signed-off-by: JeffMboya --- .github/workflows/release.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 0795377d..f4f7f728 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -6,7 +6,7 @@ name: Release Charts on: push: branches: - - main + - master jobs: release: From 94029b22a2c69b93e5bdec48c97428202463c95b Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 15:43:36 +0300 Subject: [PATCH 06/20] Update README Signed-off-by: JeffMboya check cluster status Signed-off-by: JeffMboya check cluster status Signed-off-by: JeffMboya run cluster check after creation Signed-off-by: JeffMboya update kubectl version Signed-off-by: JeffMboya update kubectl version Signed-off-by: JeffMboya update kubectl version Signed-off-by: JeffMboya update kubectl version Signed-off-by: JeffMboya update chart version Signed-off-by: JeffMboya update chart version Signed-off-by: JeffMboya update docs Signed-off-by: JeffMboya remove kubectl steps Signed-off-by: JeffMboya update ct to v2.6.1 Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 +- README.md | 71 ++++++- charts/magistrala/README.md | 349 ++++++++++++++++++++++++++----- 3 files changed, 368 insertions(+), 56 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index e68c2b73..79c0d064 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -33,7 +33,7 @@ jobs: check-latest: true - name: Set up chart-testing - uses: helm/chart-testing-action@v2.6.0 + uses: helm/chart-testing-action@v2.6.1 - name: Run chart-testing (list-changed) id: list-changed @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala + ct install --target-branch ${{ github.event.repository.default_branch }} diff --git a/README.md b/README.md index 64202165..e8280989 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,74 @@ ## DevOps -DevOps scripts for Magistrala IoT platform +Scripts for managing the Magistrala IoT platform. -## Install +## Prerequisites -Follow the instructions in [`charts`](charts) directory. +Make sure [Helm](https://helm.sh) is installed. If not, follow the [Helm documentation](https://helm.sh/docs) to get started. -Detailed documentation can be found [here](https://docs.magistrala.abstractmachines.fr/kubernetes/). +## Usage Instructions + +### Step 1: Add the Magistrala DevOps Repository + +After installing Helm, add the Magistrala DevOps Helm repository by running: + +```bash +helm repo add https://absmach.github.io/devops/ +``` + +- Replace `` with a repository alias of your choice. This alias will be used in future Helm commands. + **Example**: + + ```bash + helm repo add magistrala-devops https://absmach.github.io/devops/ + ``` + +### Step 2: Install the Magistrala Chart + +To install the Magistrala chart, run: + +```bash +helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 +``` + +- `my-magistrala`: Choose a release name for the installation. +- `magistrala-devops/magistrala`: Refers to the chart in the added repository. +- `--version 1.0.6`: Installs version 1.0.6 of the chart. + +To customize the installation, you can: +- Use the `--values` flag to provide a custom values file. +- Use the `--set` flag for inline configuration changes. + +### Step 3: Update the Repository + +If you’ve already added the Magistrala repo, update it to fetch the latest chart versions: + +```bash +helm repo update +``` + +### Step 4: Search for Charts + +To search for charts by keyword in the repository: + +```bash +helm search repo [keyword] +``` + +### Step 5: Uninstall the Magistrala Chart + +When you no longer need the chart, uninstall it by running: + +```bash +helm uninstall my-magistrala +``` + +- Replace `my-magistrala` with the release name you used during installation. This command removes all resources created by the chart and clears the release history. + +For further details, refer to the [Magistrala Kubernetes documentation](https://docs.magistrala.abstractmachines.fr/kubernetes/). ## License -[Apace-2.0](LICENSE) +This project is licensed under the [Apache-2.0](LICENSE). + +--- \ No newline at end of file diff --git a/charts/magistrala/README.md b/charts/magistrala/README.md index 12022dad..ed52d3eb 100644 --- a/charts/magistrala/README.md +++ b/charts/magistrala/README.md @@ -1,61 +1,312 @@ -## Usage +# magistrala -[Helm](https://helm.sh) must be installed to use Magistrala charts. Please refer to Helm's [documentation](https://helm.sh/docs) to get started if you haven't already. +![Version: 1.0.6](https://img.shields.io/badge/Version-1.0.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.14.0](https://img.shields.io/badge/AppVersion-0.14.0-informational?style=flat-square) -### Step 1: Adding the Magistrala DevOps Repo +Magistrala IoT Platform -Once Helm has been set up correctly, add the Magistrala DevOps repository by running the following command: +**Homepage:** -```bash -helm repo add https://absmach.github.io/devops/ -``` +## Maintainers -`` is the alias or label that you assign to the repository you are adding. It can be any name of your choice, and it will act as a reference for the repository in your subsequent Helm commands. +| Name | Email | Url | +| ---- | ------ | --- | +| drasko | | | +| dusan | | | -For example: +## Source Code -```bash -helm repo add magistrala-devops https://absmach.github.io/devops/ -``` +* -### Step 2: Installing the Chart +## Requirements -To install the Magistrala chart, run the following command: +| Repository | Name | Version | +|------------|------|---------| +| @bitnami | postgresqlbootstrap(postgresql) | 12.5.6 | +| @bitnami | postgresqlinvitations(postgresql) | 12.5.6 | +| @bitnami | postgresqlauth(postgresql) | 12.5.6 | +| @bitnami | postgresqlspicedb(postgresql) | 12.5.6 | +| @bitnami | postgresqlthings(postgresql) | 12.5.6 | +| @bitnami | postgresqlusers(postgresql) | 12.5.6 | +| @bitnami | postgresqlui(postgresql) | 12.5.6 | +| @bitnami | postgresqlcerts(postgresql) | 12.5.6 | +| @bitnami | timescaledb(postgresql) | 12.5.6 | +| @bitnami | postgresqljournal(postgresql) | 12.5.6 | +| @bitnami | redis-things(redis) | 19.6.2 | +| @hashicorp | vault(vault) | 0.28.1 | +| @jaegertracing | jaeger | 3.1.1 | +| @nats | nats | 1.2.1 | -```bash -helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 -``` +## Values -* `my-magistrala` is the release name you choose for the installation. You can modify this to suit your needs. -* `magistrala-devops/magistrala` specifies the chart name from the added repository. -* `--version 1.0.6` ensures you install the specific version of the chart. +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| adapter_coap.image | object | `{}` | | +| adapter_coap.port | int | `5683` | | +| adapter_http.httpPort | int | `8008` | | +| adapter_http.image | object | `{}` | | +| auth.accessTokenDuration | string | `"1h"` | | +| auth.adminEmail | string | `"admin@example.com"` | | +| auth.adminPassword | string | `"12345678"` | | +| auth.affinity | object | `{}` | | +| auth.grpcPort | int | `8181` | | +| auth.httpPort | int | `8189` | | +| auth.image | object | `{}` | | +| auth.invitationDuration | string | `"168h"` | | +| auth.nodeSelector | object | `{}` | | +| auth.refreshTokenDuration | string | `"24h"` | | +| auth.secret | string | `"supersecret"` | | +| auth.tolerations | object | `{}` | | +| bootstrap.enabled | bool | `true` | | +| bootstrap.encKey | string | `"randomstring"` | | +| bootstrap.eventConsumerName | string | `"EventConsumerByBootstrap"` | | +| bootstrap.httpPort | int | `9013` | | +| bootstrap.image | object | `{}` | | +| bootstrap.redisESPort | int | `6379` | | +| certs.enabled | bool | `false` | | +| certs.httpPort | int | `9019` | | +| certs.image | object | `{}` | | +| certs.logLevel | string | `"info"` | | +| certs.signCAKeyPath | string | `"/etc/ssl/certs/ca.key"` | | +| certs.signCAPath | string | `"/etc/ssl/certs/ca.crt"` | | +| certs.vault.approleRoleid | string | `""` | | +| certs.vault.approleSecret | string | `""` | | +| certs.vault.namespace | string | `""` | | +| certs.vault.thingsCertsPkiPath | string | `""` | | +| certs.vault.thingsCertsPkiRoleName | string | `""` | | +| certs.vault.url | string | `""` | | +| defaults.eventStreamURL | string | `"magistrala-nats:4222"` | | +| defaults.image.pullPolicy | string | `"IfNotPresent"` | | +| defaults.image.rootRepository | string | `"magistrala"` | | +| defaults.image.tag | string | `"latest"` | | +| defaults.jaegerCollectorPort | int | `4318` | | +| defaults.jaegerTraceRatio | int | `10` | | +| defaults.logLevel | string | `"info"` | | +| defaults.natsPort | int | `4222` | | +| defaults.replicaCount | int | `3` | | +| defaults.sendTelemetry | bool | `true` | | +| envoy.image.pullPolicy | string | `"IfNotPresent"` | | +| envoy.image.repository | string | `"envoyproxy/envoy"` | | +| envoy.image.tag | string | `"v1.31-latest"` | | +| ingress.annotations | object | `{}` | | +| ingress.enabled | bool | `true` | | +| ingress.labels | object | `{}` | | +| invitations.enabled | bool | `true` | | +| invitations.httpPort | int | `9020` | | +| invitations.image | object | `{}` | | +| jaeger.agent.enabled | bool | `false` | | +| jaeger.allInOne.enabled | bool | `false` | | +| jaeger.collector.service.otlp.grpc.name | string | `"otlp-grpc"` | | +| jaeger.collector.service.otlp.grpc.port | int | `4317` | | +| jaeger.collector.service.otlp.http.name | string | `"otlp-http"` | | +| jaeger.collector.service.otlp.http.port | int | `4318` | | +| jaeger.fullnameOverride | string | `"magistrala-jaeger"` | | +| jaeger.provisionDataStore.cassandra | bool | `true` | | +| jaeger.storage.type | string | `"memory"` | | +| journal.enabled | bool | `true` | | +| journal.httpPort | int | `9021` | | +| journal.image | object | `{}` | | +| mqtt.adapter.image.pullSecrets | object | `{}` | | +| mqtt.adapter.logLevel | string | `"debug"` | | +| mqtt.adapter.mqttPort | int | `1884` | | +| mqtt.adapter.wsPort | int | `8081` | | +| mqtt.broker.image.repository | string | `"magistrala/vernemq"` | | +| mqtt.broker.logLevel | string | `"info"` | | +| mqtt.broker.mqttPort | int | `1883` | | +| mqtt.broker.persistentVolume.size | string | `"5Gi"` | | +| mqtt.broker.wsPort | int | `8080` | | +| mqtt.enabled | bool | `true` | | +| mqtt.redisCachePort | int | `6379` | | +| mqtt.redisESPort | int | `6379` | | +| mqtt.securityContext.fsGroup | int | `10000` | | +| mqtt.securityContext.runAsGroup | int | `10000` | | +| mqtt.securityContext.runAsUser | int | `10000` | | +| nats.config.cluster.enabled | bool | `false` | | +| nats.config.cluster.replicas | int | `3` | | +| nats.config.jetstream.enabled | bool | `true` | | +| nats.config.jetstream.fileStore.enabled | bool | `true` | | +| nats.config.jetstream.fileStore.pvc.enabled | bool | `true` | | +| nats.config.jetstream.memoryStore.enabled | bool | `true` | | +| nats.config.jetstream.memoryStore.maxSize | string | `"2Gi"` | | +| nginxInternal.image.pullPolicy | string | `"IfNotPresent"` | | +| nginxInternal.image.repository | string | `"nginx"` | | +| nginxInternal.image.tag | string | `"1.19.1-alpine"` | | +| nginxInternal.mtls.intermediateCrt | string | `""` | | +| nginxInternal.mtls.tls | string | `""` | | +| postgresqlauth.database | string | `"auth"` | | +| postgresqlauth.enabled | bool | `true` | | +| postgresqlauth.global.postgresql.auth.database | string | `"auth"` | | +| postgresqlauth.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlauth.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlauth.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlauth.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlauth.host | string | `"postgresql-auth"` | | +| postgresqlauth.name | string | `"postgresql-auth"` | | +| postgresqlauth.password | string | `"magistrala"` | | +| postgresqlauth.port | int | `5432` | | +| postgresqlauth.username | string | `"magistrala"` | | +| postgresqlbootstrap.database | string | `"bootstrap"` | | +| postgresqlbootstrap.enabled | bool | `true` | | +| postgresqlbootstrap.global.postgresql.auth.database | string | `"bootstrap"` | | +| postgresqlbootstrap.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlbootstrap.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlbootstrap.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlbootstrap.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlbootstrap.host | string | `"postgresql-bootstrap"` | | +| postgresqlbootstrap.name | string | `"postgresql-bootstrap"` | | +| postgresqlbootstrap.password | string | `"magistrala"` | | +| postgresqlbootstrap.port | int | `5432` | | +| postgresqlbootstrap.username | string | `"magistrala"` | | +| postgresqlcerts.database | string | `"certs"` | | +| postgresqlcerts.enabled | bool | `true` | | +| postgresqlcerts.global.postgresql.auth.database | string | `"certs"` | | +| postgresqlcerts.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlcerts.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlcerts.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlcerts.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlcerts.host | string | `"postgresql-certs"` | | +| postgresqlcerts.name | string | `"postgresql-certs"` | | +| postgresqlcerts.password | string | `"magistrala"` | | +| postgresqlcerts.port | int | `5432` | | +| postgresqlcerts.username | string | `"magistrala"` | | +| postgresqlinvitations.database | string | `"invitations"` | | +| postgresqlinvitations.enabled | bool | `true` | | +| postgresqlinvitations.global.postgresql.auth.database | string | `"invitations"` | | +| postgresqlinvitations.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlinvitations.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlinvitations.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlinvitations.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlinvitations.host | string | `"postgresql-invitations"` | | +| postgresqlinvitations.name | string | `"postgresql-invitations"` | | +| postgresqlinvitations.password | string | `"magistrala"` | | +| postgresqlinvitations.port | int | `5432` | | +| postgresqlinvitations.username | string | `"magistrala"` | | +| postgresqljournal.database | string | `"journal"` | | +| postgresqljournal.enabled | bool | `true` | | +| postgresqljournal.global.postgresql.auth.database | string | `"journal"` | | +| postgresqljournal.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqljournal.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqljournal.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqljournal.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqljournal.host | string | `"postgresql-journal"` | | +| postgresqljournal.name | string | `"postgresql-journal"` | | +| postgresqljournal.password | string | `"magistrala"` | | +| postgresqljournal.port | int | `5432` | | +| postgresqljournal.username | string | `"magistrala"` | | +| postgresqlspicedb.database | string | `"spicedb"` | | +| postgresqlspicedb.enabled | bool | `true` | | +| postgresqlspicedb.global.postgresql.auth.database | string | `"spicedb"` | | +| postgresqlspicedb.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlspicedb.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlspicedb.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlspicedb.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlspicedb.host | string | `"postgresql-spicedb"` | | +| postgresqlspicedb.name | string | `"postgresql-spicedb"` | | +| postgresqlspicedb.password | string | `"magistrala"` | | +| postgresqlspicedb.port | int | `5432` | | +| postgresqlspicedb.username | string | `"magistrala"` | | +| postgresqlthings.database | string | `"things"` | | +| postgresqlthings.enabled | bool | `true` | | +| postgresqlthings.global.postgresql.auth.database | string | `"things"` | | +| postgresqlthings.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlthings.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlthings.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlthings.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlthings.host | string | `"postgresql-things"` | | +| postgresqlthings.name | string | `"postgresql-things"` | | +| postgresqlthings.password | string | `"magistrala"` | | +| postgresqlthings.port | int | `5432` | | +| postgresqlthings.username | string | `"magistrala"` | | +| postgresqlui.database | string | `"ui"` | | +| postgresqlui.enabled | bool | `true` | | +| postgresqlui.global.postgresql.auth.database | string | `"ui"` | | +| postgresqlui.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlui.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlui.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlui.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlui.host | string | `"postgresql-ui"` | | +| postgresqlui.name | string | `"postgresql-ui"` | | +| postgresqlui.password | string | `"magistrala"` | | +| postgresqlui.port | int | `5432` | | +| postgresqlui.username | string | `"magistrala"` | | +| postgresqlusers.database | string | `"users"` | | +| postgresqlusers.enabled | bool | `true` | | +| postgresqlusers.global.postgresql.auth.database | string | `"users"` | | +| postgresqlusers.global.postgresql.auth.password | string | `"magistrala"` | | +| postgresqlusers.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| postgresqlusers.global.postgresql.auth.username | string | `"magistrala"` | | +| postgresqlusers.global.postgresql.service.ports.postgresql | int | `5432` | | +| postgresqlusers.host | string | `"postgresql-users"` | | +| postgresqlusers.name | string | `"postgresql-users"` | | +| postgresqlusers.password | string | `"magistrala"` | | +| postgresqlusers.port | int | `5432` | | +| postgresqlusers.username | string | `"magistrala"` | | +| redis-things.cluster.enabled | bool | `false` | | +| redis-things.usePassword | bool | `false` | | +| redis-things.volumePermissions.enabled | bool | `true` | | +| spicedb.affinity | object | `{}` | | +| spicedb.datastore.engine | string | `"postgres"` | | +| spicedb.dispatch.enabled | bool | `false` | | +| spicedb.dispatch.port | int | `50053` | | +| spicedb.grpc.port | int | `50051` | | +| spicedb.grpc.presharedKey | string | `"helloworld"` | | +| spicedb.http.enabled | bool | `false` | | +| spicedb.http.port | int | `8443` | | +| spicedb.image.pullSecrets | object | `{}` | | +| spicedb.image.repository | string | `"authzed/spicedb"` | | +| spicedb.image.tag | string | `"latest"` | | +| spicedb.metrics.enabled | bool | `true` | | +| spicedb.metrics.port | int | `9090` | | +| spicedb.nodeSelector | object | `{}` | | +| spicedb.tolerations | object | `{}` | | +| things.authGrpcPort | int | `7000` | | +| things.authHttpPort | int | `9001` | | +| things.httpPort | int | `9000` | | +| things.image | object | `{}` | | +| things.redisCachePort | int | `6379` | | +| things.redisESPort | int | `6379` | | +| timescaledb.database | string | `"messages"` | | +| timescaledb.enabled | bool | `true` | | +| timescaledb.global.postgresql.auth.database | string | `"messages"` | | +| timescaledb.global.postgresql.auth.password | string | `"magistrala"` | | +| timescaledb.global.postgresql.auth.postgresPassword | string | `"magistrala"` | | +| timescaledb.global.postgresql.auth.username | string | `"magistrala"` | | +| timescaledb.global.postgresql.service.ports.postgresql | int | `5432` | | +| timescaledb.host | string | `"timescalerw"` | | +| timescaledb.image.registry | string | `"docker.io"` | | +| timescaledb.image.repository | string | `"timescale/timescaledb"` | | +| timescaledb.image.tag | string | `"latest-pg12"` | | +| timescaledb.name | string | `"timescalerw"` | | +| timescaledb.password | string | `"magistrala"` | | +| timescaledb.port | int | `5432` | | +| timescaledb.reader.enabled | bool | `true` | | +| timescaledb.reader.http.port | int | `9011` | | +| timescaledb.reader.image | object | `{}` | | +| timescaledb.username | string | `"magistrala"` | | +| timescaledb.writer.enabled | bool | `true` | | +| timescaledb.writer.http.port | int | `9012` | | +| timescaledb.writer.image | object | `{}` | | +| ui.blockKey | string | `"UtgZjr92jwRY6SPUndHXiyl9QY8qTUyZ"` | | +| ui.contentType | string | `"application/senml+json"` | | +| ui.enabled | bool | `true` | | +| ui.googleClientID | string | `""` | | +| ui.googleClientSecret | string | `""` | | +| ui.googleRedirectHostname | string | `"https://stage-domain-name"` | | +| ui.googleRedirectPath | string | `"/oauth/callback/google"` | | +| ui.googleState | string | `"somerandomstring"` | | +| ui.hashKey | string | `"5jx4x2Qg9OUmzpP5dbveWQ"` | | +| ui.image | object | `{}` | | +| ui.pathPrefix | string | `"/ui"` | | +| ui.port | int | `9095` | | +| users.adminEmail | string | `"admin@example.com"` | | +| users.adminPassword | string | `"12345678"` | | +| users.allowSelfRegister | bool | `true` | | +| users.deleteAfter | string | `"720h"` | | +| users.deleteInterval | string | `"24h"` | | +| users.grpcPort | int | `7001` | | +| users.httpPort | int | `9002` | | +| users.image | object | `{}` | | +| users.passwordRegex | string | `"^.{8,}$"` | | +| users.secretKey | string | `"secretKey"` | | +| users.tokenResetEndpoint | string | `"/reset-request"` | | +| vault.enabled | bool | `false` | | -* To override values in the chart, use either the `--values` flag and pass in a file or use the `--set` flag and pass configuration from the command line. - -### Step 3: Updating the Repository - -If you had already added this repo earlier, run the following command to retrieve the latest versions of the packages: - -```bash -helm repo update -``` - -### Step 4: Searching for Charts - -You can search the repository for a keyword in charts using the following command: - -```bash -helm search repo [keyword] [flags] -``` - -### Step 5: Uninstalling the Chart - -To uninstall the chart when you no longer need it, run the following command: - -```bash -helm uninstall my-magistrala [flags] -``` - -Make sure to replace `my-magistrala` with your actual release name if you changed it during installation. The command removes all of the resources associated with the last release of the chart as well as the release history, freeing it up for future use. - ---- \ No newline at end of file From 896f893425351e2a384e1f3eca53267dcfef45e8 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 17:30:09 +0300 Subject: [PATCH 07/20] replace command line arguments with config file Signed-off-by: JeffMboya use config file for ct Signed-off-by: JeffMboya use config file for ct Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 6 ++++-- ct.yaml | 6 ++++++ 2 files changed, 10 insertions(+), 2 deletions(-) create mode 100644 ct.yaml diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 79c0d064..c3293dc1 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -27,6 +27,8 @@ jobs: helm repo add hashicorp https://helm.releases.hashicorp.com helm repo update + # Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and + # yamllint (https://github.com/adrienverge/yamllint) which require Python - uses: actions/setup-python@v5 with: python-version: "3.x" @@ -46,7 +48,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala + ct lint --config ct.yaml - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +57,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} + ct install --config ct.yaml diff --git a/ct.yaml b/ct.yaml new file mode 100644 index 00000000..3581b939 --- /dev/null +++ b/ct.yaml @@ -0,0 +1,6 @@ +# See https://github.com/helm/chart-testing#configuration +remote: origin +target-branch: master +chart-dirs: + - charts/magistrala +helm-extra-args: --timeout 600s \ No newline at end of file From 60a2a1bc523313fdd693bd0441b014ff48275a00 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 18:15:50 +0300 Subject: [PATCH 08/20] add arguments to ct install Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 6 ++---- ct.yaml | 6 ------ 2 files changed, 2 insertions(+), 10 deletions(-) delete mode 100644 ct.yaml diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index c3293dc1..12fa9c72 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -27,8 +27,6 @@ jobs: helm repo add hashicorp https://helm.releases.hashicorp.com helm repo update - # Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and - # yamllint (https://github.com/adrienverge/yamllint) which require Python - uses: actions/setup-python@v5 with: python-version: "3.x" @@ -48,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --config ct.yaml + ct lint --target-branch ${{ github.event.repository.default_branch }} --remote master --chart-dirs charts/magistrala - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -57,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --config ct.yaml + ct install --target-branch ${{ github.event.repository.default_branch }} --remote master --chart-dirs charts/magistrala diff --git a/ct.yaml b/ct.yaml deleted file mode 100644 index 3581b939..00000000 --- a/ct.yaml +++ /dev/null @@ -1,6 +0,0 @@ -# See https://github.com/helm/chart-testing#configuration -remote: origin -target-branch: master -chart-dirs: - - charts/magistrala -helm-extra-args: --timeout 600s \ No newline at end of file From 14b48a1702489064e74148b005f29825255decf7 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 18:17:35 +0300 Subject: [PATCH 09/20] add arguments to ct install Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 12fa9c72..6450e1b9 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --target-branch ${{ github.event.repository.default_branch }} --remote master --chart-dirs charts/magistrala + ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --remote master --chart-dirs charts/magistrala + ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala From c8e3015fc58f47715feed409d6929c03610fe188 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 21:07:55 +0300 Subject: [PATCH 10/20] add config options Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 6450e1b9..f81542fb 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala + ct lint --remote upstream --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala + ct install --remote upstream --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true --helm-extra-args '--timeout 3600s' From 6ae9cf8e72f31ebadb18408cd830a41c7a0aab73 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 21:10:38 +0300 Subject: [PATCH 11/20] replace upstream with remote url Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index f81542fb..dfefec14 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --remote upstream --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true + ct lint --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --remote upstream --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true --helm-extra-args '--timeout 3600s' + ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true --helm-extra-args '--timeout 3600s' From 0a696a630517884972c3b2eb87a1da98d177dda8 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 21:13:28 +0300 Subject: [PATCH 12/20] replace upstream with remote url Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index dfefec14..04630e93 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true --helm-extra-args '--timeout 3600s' + ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --helm-extra-args '--timeout 3600s' From 3cb91f9c33eb1d1abcbb5ffd7aa5fae2ba8191f2 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 21:57:00 +0300 Subject: [PATCH 13/20] replace upstream with remote url Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 04630e93..50e94f91 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --check-version-increment true + ct lint --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts --check-version-increment true - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --charts charts/magistrala --helm-extra-args '--timeout 3600s' + ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file From d53b7944534598f8923adf80b35a5dbe2ac2d269 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 22:00:33 +0300 Subject: [PATCH 14/20] fix target branch Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 50e94f91..d3e0f4e1 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts --check-version-increment true + ct lint --remote https://github.com/absmach/devops/tree --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts --check-version-increment true - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --remote https://github.com/absmach/devops --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file + ct install --remote https://github.com/absmach/devops/tree --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file From 43ccbf86573c888d56c6852c6e91678360e9cb26 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 22:02:59 +0300 Subject: [PATCH 15/20] fix target branch Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index d3e0f4e1..a6165c92 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -46,7 +46,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --remote https://github.com/absmach/devops/tree --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts --check-version-increment true + ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -55,4 +55,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --remote https://github.com/absmach/devops/tree --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file + ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file From 103f4c73ca13d0fa130833ed74ceaab13883b91a Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 22:39:12 +0300 Subject: [PATCH 16/20] add step to fetch and install those dependencies Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index a6165c92..8b6f9dd8 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -19,7 +19,7 @@ jobs: with: version: v3.15.4 - - name: Add Helm dependencies + - name: Add Helm repositories and update run: | helm repo add nats https://nats-io.github.io/k8s/helm/charts/ helm repo add jaegertracing https://jaegertracing.github.io/helm-charts @@ -27,6 +27,12 @@ jobs: helm repo add hashicorp https://helm.releases.hashicorp.com helm repo update + - name: Update Helm dependencies + run: | + helm dependency update charts/magistrala + + # Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and + # yamllint (https://github.com/adrienverge/yamllint) which require Python - uses: actions/setup-python@v5 with: python-version: "3.x" @@ -55,4 +61,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts \ No newline at end of file + ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts From 26af3d7a8e101a5fbe5920aed1dbdb09bda7e74a Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 22:54:12 +0300 Subject: [PATCH 17/20] process all charts Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 2 +- .github/workflows/release.yaml | 4 +++ README.md | 52 ++++++++++++++++++++++++-------- 3 files changed, 44 insertions(+), 14 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index 8b6f9dd8..b3828566 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -61,4 +61,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts + ct install --target-branch ${{ github.event.repository.default_branch }} --all diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index f4f7f728..95e740e0 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -39,6 +39,10 @@ jobs: helm repo add nats https://nats-io.github.io/k8s/helm/charts/ helm repo update + - name: Update Helm dependencies + run: | + helm dependency update charts/magistrala + - name: Run chart-releaser uses: helm/chart-releaser-action@v1.6.0 env: diff --git a/README.md b/README.md index e8280989..2c4fe1a0 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,40 @@ helm repo add https://absmach.github.io/devops/ helm repo add magistrala-devops https://absmach.github.io/devops/ ``` -### Step 2: Install the Magistrala Chart +### Step 2: Add External Helm Repositories for Dependencies + +Magistrala charts rely on several external charts. To ensure these dependencies can be resolved, add the required Helm repositories: + +```bash +helm repo add nats https://nats-io.github.io/k8s/helm/charts/ +helm repo add jaegertracing https://jaegertracing.github.io/helm-charts +helm repo add bitnami https://charts.bitnami.com/bitnami +helm repo add hashicorp https://helm.releases.hashicorp.com +``` + +This step is important to ensure all chart dependencies are properly fetched. + +### Step 3: Update Helm Repositories + +After adding the required repositories, update your local repository cache to fetch the latest chart versions: + +```bash +helm repo update +``` + +This ensures that your Helm client is aware of the latest available versions of the charts in these repositories. + +### Step 4: Update Chart Dependencies + +If your Helm chart has dependencies specified in `Chart.yaml`, you need to resolve and download them by running: + +```bash +helm dependency update charts/magistrala +``` + +This downloads the necessary dependencies into the `charts/magistrala/charts/` directory so that the Helm chart can install correctly. + +### Step 5: Install the Magistrala Chart To install the Magistrala chart, run: @@ -31,23 +64,16 @@ To install the Magistrala chart, run: helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 ``` -- `my-magistrala`: Choose a release name for the installation. -- `magistrala-devops/magistrala`: Refers to the chart in the added repository. +- `my-magistrala`: Choose a release name for the installation. +- `magistrala-devops/magistrala`: Refers to the chart in the added repository. - `--version 1.0.6`: Installs version 1.0.6 of the chart. To customize the installation, you can: + - Use the `--values` flag to provide a custom values file. - Use the `--set` flag for inline configuration changes. -### Step 3: Update the Repository - -If you’ve already added the Magistrala repo, update it to fetch the latest chart versions: - -```bash -helm repo update -``` - -### Step 4: Search for Charts +### Step 6: Search for Charts To search for charts by keyword in the repository: @@ -55,7 +81,7 @@ To search for charts by keyword in the repository: helm search repo [keyword] ``` -### Step 5: Uninstall the Magistrala Chart +### Step 7: Uninstall the Magistrala Chart When you no longer need the chart, uninstall it by running: From 0cbcb04e06fdfe466ae90405671167c71f97c2f4 Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Tue, 10 Sep 2024 23:03:42 +0300 Subject: [PATCH 18/20] update chart dir Signed-off-by: JeffMboya --- .github/workflows/lint-test.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/lint-test.yaml b/.github/workflows/lint-test.yaml index b3828566..e872aaf7 100644 --- a/.github/workflows/lint-test.yaml +++ b/.github/workflows/lint-test.yaml @@ -52,7 +52,7 @@ jobs: - name: Run chart-testing (lint) if: steps.list-changed.outputs.changed == 'true' run: | - ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts + ct lint --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala/charts - name: Create kind cluster if: steps.list-changed.outputs.changed == 'true' @@ -61,4 +61,4 @@ jobs: - name: Run chart-testing (install) if: steps.list-changed.outputs.changed == 'true' run: | - ct install --target-branch ${{ github.event.repository.default_branch }} --all + ct install --target-branch ${{ github.event.repository.default_branch }} --chart-dirs charts/magistrala/charts From dd181cf439f8ada044586df2e7f6a8035e8bb70d Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Wed, 11 Sep 2024 08:37:00 +0300 Subject: [PATCH 19/20] update README Signed-off-by: JeffMboya --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2c4fe1a0..ad481aa7 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ This ensures that your Helm client is aware of the latest available versions of ### Step 4: Update Chart Dependencies -If your Helm chart has dependencies specified in `Chart.yaml`, you need to resolve and download them by running: +Since the Helm chart has dependencies specified in `Chart.yaml`, you need to resolve and download them by running: ```bash helm dependency update charts/magistrala @@ -64,7 +64,7 @@ To install the Magistrala chart, run: helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 ``` -- `my-magistrala`: Choose a release name for the installation. +- `my-magistrala`: release name for the installation. You can use any name of your choice - `magistrala-devops/magistrala`: Refers to the chart in the added repository. - `--version 1.0.6`: Installs version 1.0.6 of the chart. From 98c3883b2db508107ab8ad4b0a99f1db23d2e8aa Mon Sep 17 00:00:00 2001 From: JeffMboya Date: Wed, 11 Sep 2024 13:26:33 +0300 Subject: [PATCH 20/20] Update README Signed-off-by: JeffMboya --- README.md | 94 +++++++++++++++++-------------------------------------- 1 file changed, 28 insertions(+), 66 deletions(-) diff --git a/README.md b/README.md index ad481aa7..6670588a 100644 --- a/README.md +++ b/README.md @@ -1,100 +1,62 @@ ## DevOps -Scripts for managing the Magistrala IoT platform. +Scripts for managing the Magistrala IoT platform. For installation instructions, refer to [Magistrala Kubernetes Documentation](https://docs.magistrala.abstractmachines.fr/kubernetes/). -## Prerequisites +### Autogenerating Helm Chart Documentation -Make sure [Helm](https://helm.sh) is installed. If not, follow the [Helm documentation](https://helm.sh/docs) to get started. +The documentation for Magistrala Helm charts in `charts/magistrala/README.md` is generated using `helm-docs`, which extracts metadata from `Chart.yaml` and `values.yaml`. To update the documentation after changes, follow these steps: -## Usage Instructions +### Prerequisites -### Step 1: Add the Magistrala DevOps Repository +Before starting, ensure the following tools are installed: -After installing Helm, add the Magistrala DevOps Helm repository by running: +1. **Helm** + Make sure [Helm](https://helm.sh) is installed on your system. If not, follow the [Helm installation guide](https://helm.sh/docs) to get started. -```bash -helm repo add https://absmach.github.io/devops/ -``` +2. **Helm Docs Tool** + The documentation for the Magistrala Helm charts is autogenerated using the `helm-docs` tool. To install `helm-docs`, use the following command: -- Replace `` with a repository alias of your choice. This alias will be used in future Helm commands. - **Example**: + ```bash + go install github.com/norwoodj/helm-docs/cmd/helm-docs@latest + ``` - ```bash - helm repo add magistrala-devops https://absmach.github.io/devops/ - ``` + If Go is not installed, follow the [Go installation guide](https://golang.org/doc/install). -### Step 2: Add External Helm Repositories for Dependencies +### Step 1: Navigate to Your Project Directory -Magistrala charts rely on several external charts. To ensure these dependencies can be resolved, add the required Helm repositories: +First, move to the directory where the Helm charts are stored. For this project, the command would be: ```bash -helm repo add nats https://nats-io.github.io/k8s/helm/charts/ -helm repo add jaegertracing https://jaegertracing.github.io/helm-charts -helm repo add bitnami https://charts.bitnami.com/bitnami -helm repo add hashicorp https://helm.releases.hashicorp.com +cd devops ``` -This step is important to ensure all chart dependencies are properly fetched. - -### Step 3: Update Helm Repositories +### Step 2: Run the `helm-docs` Command -After adding the required repositories, update your local repository cache to fetch the latest chart versions: +Generate or update the documentation for your Helm charts by running: ```bash -helm repo update +helm-docs ``` -This ensures that your Helm client is aware of the latest available versions of the charts in these repositories. - -### Step 4: Update Chart Dependencies - -Since the Helm chart has dependencies specified in `Chart.yaml`, you need to resolve and download them by running: +This command will parse the charts in the `charts` directory and update the `charts/magistrala/README.md` file. A typical successful run looks like this: ```bash -helm dependency update charts/magistrala +INFO[2024-09-11T11:34:20+03:00] Found Chart directories [charts/magistrala] +INFO[2024-09-11T11:34:20+03:00] Generating README Documentation for chart charts/magistrala ``` -This downloads the necessary dependencies into the `charts/magistrala/charts/` directory so that the Helm chart can install correctly. - -### Step 5: Install the Magistrala Chart +### Step 3: Commit and Push the Changes -To install the Magistrala chart, run: +After `helm-docs` has updated the documentation, review the changes, and then commit and push them to your Git repository: ```bash -helm install my-magistrala magistrala-devops/magistrala --version 1.0.6 +git add charts/magistrala/README.md +git commit -m "Update Helm chart documentation" +git push origin ``` -- `my-magistrala`: release name for the installation. You can use any name of your choice -- `magistrala-devops/magistrala`: Refers to the chart in the added repository. -- `--version 1.0.6`: Installs version 1.0.6 of the chart. - -To customize the installation, you can: - -- Use the `--values` flag to provide a custom values file. -- Use the `--set` flag for inline configuration changes. - -### Step 6: Search for Charts - -To search for charts by keyword in the repository: - -```bash -helm search repo [keyword] -``` - -### Step 7: Uninstall the Magistrala Chart - -When you no longer need the chart, uninstall it by running: - -```bash -helm uninstall my-magistrala -``` - -- Replace `my-magistrala` with the release name you used during installation. This command removes all resources created by the chart and clears the release history. - -For further details, refer to the [Magistrala Kubernetes documentation](https://docs.magistrala.abstractmachines.fr/kubernetes/). +Replace `` with the branch you are working on. ## License This project is licensed under the [Apache-2.0](LICENSE). - ---- \ No newline at end of file