Getting valueFrom.secretKeyRef.name invalid value

[NChittimalla]

@NAjustin can you help with this?

Deployment.apps "airbyte-workload-launcher" is invalid: [spec.template.spec.containers[0].env[62].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*'),

Deployment.apps "airbyte-server" is invalid: [spec.template.spec.containers[0].env[62].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*'),

Deployment.apps "worker" is invalid: [spec.template.spec.containers[0].env[62].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*'),

My values.yaml file and secrets file are created successfully but unable to know why iam getting this invalid value?? My pods are succesfully running and only one pod Temporal is Failed and airbyte-server, worker, workload-launcher 3 pods are not created.

[justbeez]

@NChittimalla Can you please post a version of your values.yaml with any sensitive values redacted? And also provide the Airbyte app and helm chart or abctl versions you're using.

There aren't that many config options that take a domain/subdomain value, so my initial thoughts are global.airbyteURl, global.database.host (or a value in a secret referenced by global.database.hostSecretKey), or something related to the ingress like webapp.ingress.hosts.host. But there are obviously a bunch of other options depending on which integrations you're using.

[NChittimalla]

@justbeez @NAjustin Please find the attachment of my values.yaml file.
values.pdf

Iam using Helm chart to deploy my application in Amazon EKS with external database and storage.

Please find Helm Versions :
versions.pdf

[abuchanan-airbyte]

I think you might be missing the secretsManagerSecretName value. Looking at the chart for airbyte-workload-launcher: https://github.com/airbytehq/airbyte-platform/blob/4351d1404b7a60a5a716bb1af6d39eada4c1471a/charts/airbyte-workload-launcher/templates/deployment.yaml#L342-L351

- name: AWS_SECRET_MANAGER_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: {{ include "airbyte.secretStoreName" .Values.global.secretsManager.secretsManagerSecretName }}
              key: {{ include "airbyte.awsSecretManagerAccessKeyIdSecretKey" .Values.global.secretsManager.awsSecretManager.accessKeyIdSecretKey }}
        - name: AWS_SECRET_MANAGER_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: {{ include "airbyte.secretStoreName" .Values.global.secretsManager.secretsManagerSecretName }}
              key: {{ include "airbyte.awsSecretManagerSecretAccessKeySecretKey" .Values.global.secretsManager.awsSecretManager.secretAccessKeySecretKey }}

The error secretKeyRef.name: Invalid value: "" is saying that the secret key ref name field is empty (and then the RFC domain name part is just super confusing). I haven't found any default values for this in the charts, so you might try providing this in your values file. Looks like the path is global.secretsManager.secretsManagerSecretName

[NChittimalla]

I have added it but even though iam getting the same error.

[marcosmarxm]

@NChittimalla please share the complete values.yaml file
My recommendation to debug the problem, try to remove all components and start adding each of them to find what config has problem.

[NChittimalla]

values.pdf

Here is my complete values.yaml file.

[NChittimalla]

I cleared it, Thanks everyone for the help.

I have added external database and logs access key, which is now working fine, which i got to know from _helpers.tpl file.

logs:
accessKey:
existingSecret: airbyte-config-secrets
existingSecretKey: AWS_ACCESS_KEY_ID
secretKey:
existingSecret: airbyte-config-secrets
existingSecretKey: AWS_SECRET_ACCESS_KEY

externalDatabase:
host: production-ts-airbyte-database.cfumy80m4m8i.us-east-2.rds.amazonaws.com
port: 5432
database: airbyte_db
user: airbytedb
existingSecret: airbyte-config-secrets
existingSecretPasswordKey: DATABASE_PASSWORD
jdbcUrl: jdbc:postgresql://production-ts-airbyte-database.cfumy80m4m8i.us-east-2.rds.amazonaws.com:5432/postgres?ssl=true&sslmode=require

Now my whole file looks like below:

values.pdf

[bambangraharjo]

I'm facing same error:

  ERROR   Failed to install airbyte/airbyte Helm Chart                                                                                                                                                      
  ERROR   Unable to install Airbyte locally                                                                                                                                                                 
  ERROR   unable to install airbyte chart: unable to install helm: 3 errors occurred:
          	* Deployment.apps "airbyte-abctl-workload-launcher" is invalid: [spec.template.spec.containers[0].env[57].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.template.spec.containers[0].env[58].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]
          	* Deployment.apps "airbyte-abctl-server" is invalid: [spec.template.spec.containers[0].env[42].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.template.spec.containers[0].env[43].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]
          	* Deployment.apps "airbyte-abctl-worker" is invalid: [spec.template.spec.containers[0].env[44].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.template.spec.containers[0].env[45].valueFrom.secretKeyRef.name: Invalid value: "": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]

This is my values.yaml:

postgresql:
  enabled: false

global:
  database:
    secretName: "airbyte-config-secrets"
    host: "rds-postgres-13-16.somestring.some-region-1.rds.amazonaws.com"
    port: "5432"
    database: "airbyte_db"
    user: "airbyte_user"
    passwordSecretKey: "database-password"


  storage:
    type: "S3"
    storageSecretName: airbyte-config-secrets # Name of your Kubernetes secret.
    bucket: ## S3 bucket names that you've created. We recommend storing the following all in one bucket.
      log: some.bucket.in.s3
      state: some.bucket.in.s3
      workloadOutput: some.bucket.in.s3
    s3:
      region: "some-region-1" ## e.g. us-east-1
      authenticationType: credentials ## Use "credentials" or "instanceProfile"

and this is my secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
type: Opaque
stringData:
  database-password: SomePassword
  # AWS S3 Secrets
  s3-access-key-id: SomeS3AccessKeyID
  s3-secret-access-key: SomeS3SecretAccessKey

External database: Postgres 13.16
Install using abctl ver v0.13.1 and v0.16.1

[benbauer-draeger]

same issue as @bambangraharjo after the upgrade

[jasperchan]

@benbauer-draeger @bambangraharjo the S3 documentation is incorrect, if you're on the latest, use secretName instead of storageSecretName:

global:
  storage:
    type: "S3"
     secretName: airbyte-config-secrets

[bambangraharjo]

It works, thank you @jasperchan .

[benbauer-draeger]

Thanks!