[MongoDB Persistence] configuration via cmd or envs not possible

[sirchnik]

Description

Configuring FA³ST-Service mongoDB persistence is not possible by environment variables or via command line arguments in k8s.

This is important because mongoDB connection string should be a secret in k8s. A workaround is to use the secret as a volume for the entire config file but that's not what it should be and is too complicated.

Reproduction

k8s manifests

apiVersion: v1
kind: ConfigMap
metadata:
  name: faaast-config
  namespace: default
data:
  config.json: |
    {
      "core": {
        "requestHandlerThreadPoolSize": 2
      },
      "endpoints": [
        {
          "@class": "de.fraunhofer.iosb.ilt.faaast.service.endpoint.http.HttpEndpoint",
          "sniEnabled": false,
          "sslEnabled": false,
          "port": 8080
        }
      ],
      "messageBus": {
        "@class": "de.fraunhofer.iosb.ilt.faaast.service.messagebus.internal.MessageBusInternal"
      }
    }

---

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  # password
  password: cGFzc3dvcmQ=
  # mongodb://root:password@mongodb-service
  connectionString: bW9uZ29kYjovL3Jvb3Q6cGFzc3dvcmRAbW9uZ29kYi1zZXJ2aWNl
  # DEBUG
  debug: VFJBQ0U=


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: faaast-deployment
  labels:
    app: faaast
spec:
  selector:
    matchLabels:
      app: faaast
  template:
    metadata:
      labels:
        app: faaast
    spec:
      containers:
        - name: faaast
          image: fraunhoferiosb/faaast-service:1.2.0
          command:
            - java
            - -jar
            - starter.jar
            - --loglevel-faaast=$(DEBUG)
            # faaast says null="mongodb://root:$(PASSWORD)@mongodb-service"
            # - persistence.connectionString="mongodb://root:$(PASSWORD)@mongodb-service"
            # - persistence.connectionString=$(CONNECTION_STRING)
            # - persistence.database=faaast-database
            # - persistence.@class=de.fraunhofer.iosb.ilt.faaast.service.persistence.mongo.PersistenceMongo
          volumeMounts:
            - name: config-volume
              mountPath: /app/resources/config.json
              subPath: config.json
              readOnly: true
          env:
            - name: JAVA_TOOL_OPTIONS
              value: "-Xmx850m -XX:ActiveProcessorCount=2"

            # is not used
            - name: faaast_config_extension_persistence_@class
              value: de.fraunhofer.iosb.ilt.faaast.service.persistence.mongo.PersistenceMongo
            - name: faaast_config_extension_persistence_connectionString
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: connectionString
            - name: faaast_config_extension_persistence_database
              value: faaast-database

            - name: faaast_config
              value: /app/resources/config.json
            - name: DEBUG
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: debug
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: password
            - name: CONNECTION_STRING
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: connectionString
          resources:
            limits:
              memory: "2500Mi"
              cpu: "2000m"
            requests:
              memory: "800Mi"
              cpu: "1000m"
          ports:
            - containerPort: 8080

      volumes:
        - name: config-volume
          configMap:
            name: faaast-config
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: faaast-hpa
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: faaast-deployment
  minReplicas: 1
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 75

---
apiVersion: v1
kind: Service
metadata:
  name: faaast-service
spec:
  type: ClusterIP
  selector:
    app: faaast
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: faaast-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - host: service.faaast.test
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: faaast-service
                port:
                  number: 8080

mongodb k8s manifest

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongodb
  labels:
    app: mongodb
spec:
  serviceName: "mongodb"
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
        - name: mongodb
          image: mongo:latest
          ports:
            - containerPort: 27017
          env:
            - name: GLIBC_TUNABLES
              value: "glibc.pthread.rseq=0"
            #  /etc/sysctl.d/mongo-k8s.conf
            #  vm.max_map_count = 262144
            #  vm.swappiness = 1

            - name: MONGO_INITDB_ROOT_USERNAME
              value: root
            - name: MONGO_INITDB_ROOT_PASSWORD
              value: password
            - name: MONGO_DATA_DIR
              value: "/data/db"
            - name: MONGO_LOG_DIR
              value: "/var/log/mongodb"
          volumeMounts:
            - name: mongodb-data
              mountPath: /data/db
            - name: mongodb-logs
              mountPath: /var/log/mongodb
  volumeClaimTemplates:
    - metadata:
        name: mongodb-data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 20Gi
    - metadata:
        name: mongodb-logs
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 5Gi

---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
spec:
  ports:
    - port: 27017
  clusterIP: None
  selector:
    app: mongodb

Output

Envs

2025-01-13 11:47:58 [INFO] Model: empty (default) (de.fraunhofer.iosb.ilt.faaast.service.starter.App)
2025-01-13 11:47:58 Model validation is disabled when using empty model
2025-01-13 11:47:58 [INFO] Model validation is disabled when using empty model (de.fraunhofer.iosb.ilt.faaast.service.starter.App)
2025-01-13 11:47:58 [ERROR] Unresolvable environment variable found 'null'. (de.fraunhofer.iosb.ilt.faaast.service.starter.ExecutionExceptionHandler)

args

2025-01-13 11:51:36 Overriding config parameter: 
                       null="mongodb://root:password@mongodb-service" [CLI]
                       null=faaast-database [CLI]
                       persistence.@class=de.fraunhofer.iosb.ilt.faaast.service.persistence.mongo.PersistenceMongo [CLI]
2025-01-13 11:51:36 [INFO] Overriding config parameter: 
                       null="mongodb://root:password@mongodb-service" [CLI]
                       null=faaast-database [CLI]
                       persistence.@class=de.fraunhofer.iosb.ilt.faaast.service.persistence.mongo.PersistenceMongo [CLI] (de.fraunhofer.iosb.ilt.faaast.service.starter.App)
2025-01-13 11:51:36 [ERROR] updating property failed (key: null, value: "mongodb://root:password@mongodb-service") (de.fraunhofer.iosb.ilt.faaast.service.starter.ExecutionExceptionHandler)

Command line arguments and environment variables are scrabbled to null.

Additional Context

• I am reporting this issue solely for testing purposes, as I am currently verifying which AAS implementations are functioning. Therefore, I don't need a fix at this time.

[mjacoby]

This is actually the intended behavior as changes to the configuration via CLI and also environment variables are intended to only change existing properties and not add new properties. So currently the only way to achieve your goal is to mount a custom configuration file.

I understand your intention and I agree that adding new properties via CLI/env might make the life easier for developers, especially in such scenarios as yours. We are consider adding this functionality in the future but cannot give any guarantees or timeline if/when this will happen. I therefore labelled this issue as enhancement