Cannot create RDS DBInstance when autogeneratePassword - clean instance

[JaniszM]

What happened?

Using ArgoCD.

Creating a clean DB, without a restoration point, and with the master password being generated.

Got error:

cannot create DBInstance in AWS: InvalidParameterValue: Invalid master password status code: 400, request id: 

Seems the problem might be the silent error ignorance for missing k8s Secrets (masterUserPasswordSecretRef).

provider-aws/pkg/controller/docdb/dbcluster/setup.go

Line 229 in f759905

if resource.IgnoreNotFound(err) != nil {

(thanks @sysradium for that check)

Creating the secret and the key didn't solve the problem, the password was not generated. I have provided a password by hand.

To get the Instance I had to create the secret and put a valid password over there.

How can we reproduce it?

This can help, my resource params:

forProvider:
    allocatedStorage: 20
    allowMajorVersionUpgrade: false
    autogeneratePassword: true
    dbInstanceClass: db.t3.micro
    dbParameterGroupName: db-mysql-group
    dbParameterGroupNameRef:
      name: db-mysql-group
    dbSubnetGroupName: db-mysql-sng
    dbSubnetGroupNameRef:
      name: db-mysql-sng
    dbSubnetGroupNameSelector:
      matchControllerRef: true
      matchLabels:
        environment: dev
        instance: db-mysql
    enableIAMDatabaseAuthentication: true
    engine: mysql
    engineVersion: 5.7.42
    masterUserPasswordSecretRef:
      key: password
      name: db-mysql-pass
      namespace: dev
    masterUsername: admin
    multiAZ: false
    region: eu-central-1
    skipFinalSnapshot: true
    storageType: gp2
    tags:
      - key: Creator
        value: Crossplane
      - key: Name
        value: db-mysql
      - key: Environment
        value: dev
    vpcSecurityGroupIDRefs:
      - name: db-mysql-sg
    vpcSecurityGroupIDSelector:
      matchControllerRef: true
      matchLabels:
        environment: dev
        instance: db-mysql
    vpcSecurityGroupIDs:
      - sg-

What environment did it happen in?

Crossplane version: helm.sh/chart=crossplane-1.13.2
provider-aws: v0.43.1
EKS/k8s: v1.28

[sysradium]

We tried different combinations of secrets. No data:

apiVersion: v1
kind: Secret
metadata:
  name: db-mysql-somedb-demo-master-pass
  namespace: environments
type: Opaque

Empty data object:

apiVersion: v1
kind: Secret
data:
metadata:
  name: db-mysql-somedb-demo-master-pass
  namespace: environments
type: Opaque

Null password:

apiVersion: v1
kind: Secret
data:
  password:
metadata:
  name: db-mysql-somedb-demo-master-pass
  namespace: environments
type: Opaque

Empty password:

apiVersion: v1
kind: Secret
data:
  password: ""
metadata:
  name: db-mysql-somedb-demo-master-pass
  namespace: environments
type: Opaque

Nothing worked till we actually set it to some base64 representation of a pw.

At first I suspected that the problem is in a way api-machinery checks for NotFound error, which @JaniszM cited. But now it seems the problem is a bit different, since created secret did not solve a problem until it was actually set to something.

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[JaniszM]

/fresh

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[sysradium]

/fresh

[MisterMX]

I think for DBInstances the password is only generated if the spec.forProvider.autogeneratePassword: true AND spec.forProvider.masterUserPasswordSecretRef: null because only then the controller assumes that a password does not exist. Otherwise he will pull it from the referenced secret.