[Bug]:aws.rds. provider recouncile loop triggered by setting default parameter explictly

[momoXD007]

Is there an existing issue for this?

• I have searched the existing issues

Affected Resource(s)

rds.aws.upbound.io/v1beta1 - ParameterGroup

Resource MRs required to reproduce the bug

No response

Steps to Reproduce

Install provider, provision a Postgres database, with default parameter explicitly set

What happened?

Hello,
we are using Crossplane to provison sql and postgres databases and we believe we found a bug. An example claim we had looked like this:

[...]
spec:
  [...]
  resourceConfig:
    allowMajorVersionUpgrade: true
    applyImmediately: true
    autoMinorVersionUpgrade: true
    autogeneratePassword: true
    dbName: dbNameHere
    deleteAutomatedBackups: true
    deletionPolicy: Delete
    engine: postgres
    engineVersion: '15'
    iamAuthentication: false
    instanceClass: db.t4g.small
    iops: 3000
    kmsKeyId: >-
      arn:aws:kms:eu-central-1:<someaccout_id>:key/<some key>
    maintenanceWindow: 'Mon:00:00-Mon:03:30'
    maxAllocatedStorage: 0
    multiAz: false
    parameterFamilyGroup: postgres15
    parameterGroup:
      - applyMethod: immediate
        name: log_checkpoints
        value: '1'
    passwordSecretRef:
      key: password
      name: semantic-recommender-rds-secret
      namespace: default
    performanceInsightsEnabled: false
    skipFinalSnapshot: true
    storageType: gp3
    tags:
      Prometheus-Export-Enabled: 'Yes'
      team: exampleTeam

We noted that Crossplane was constantly reconciling the DB ParameterGroup for this claim. Crossplane tried to change the parameter “log_checkpoints”. Manually checking this parameter in aws console confirmed that the parameter is by default ‘1’ as this is the engine default.
So in the aws_console the UI shows it like this:

Name: | Value: | Source:
log_checkpoints | 1 | Engine default

so updating this parameter is useless as it already has the right value by default still Crossplane tries non-stop.
As an experiment we changed it 0 in our claim
This resulted in the following:

Name: | Value: | Source:
log_checkpoints | 0 | modified

the reouncilitation loop finished.
Now the interesting part we change it back to 1 in our claim:
Crossplane recounciles and we get the following in aws-console:

Name: | Value: | Source:
log_checkpoints | 1 | modified

The recounciliation is finished and does not happen again.

So this lead to the following theory:
The Crossplane provider for rds or more specifically for ParameterGroups enters an infinite reconciliation loop whenever a parameter is explictly set to the already existing default value. We suspect this has to do with the Source field and/or the fact that the aws-internals probably ignores request to change the parameter when it already does have the correct value.
But as said this a theory and we don't have solid proof.

Relevant Error Output Snippet

No response

Crossplane Version

1.16.0

Provider Version

1.9.0

Kubernetes Version

v1.27.15-eks-db838b0

Kubernetes Distribution

EKS

Additional Info

#1419 -> This issue seems similar, maybe the root-cause is the same