Skip to content

[release-4.19] OCPBUGS-56119: error from generateAndValidateRenderedMachineConfig function can be misleading #5044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

dkhater-redhat
Copy link
Contributor

@dkhater-redhat dkhater-redhat commented May 13, 2025

What I did
Fixes: #OCPBUGS-56119
Fix misleading error in generateAndValidateRenderedMachineConfig

When a full-render reconciliation fails, we now iterate through the
base MachineConfigs one by one—to
pinpoint the exact config that introduced the irreconcilable change.
If a partial render fails, we report that config’s name and its error,
instead of incorrectly blaming an earlier override. We still fall back
to the original combined error behavior if no single config produces
the full-render failure.

How to verify it
See https://issues.redhat.com/browse/OCPBUGS-54692

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels May 13, 2025
@openshift-ci-robot
Copy link
Contributor

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-56119, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.19.0) matches configured target version for branch (4.19.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-54697 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-54697 targets the "4.20.0" version, which is one of the valid target versions: 4.20.0
  • bug has dependents

Requesting review from QA contact:
/cc @sergiordlr

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

What I did
Fixes: #OCPBUGS-56119
Fix misleading error in generateAndValidateRenderedMachineConfig

When a full-render reconciliation fails, we now iterate through the
base MachineConfigs one by one—to
pinpoint the exact config that introduced the irreconcilable change.
If a partial render fails, we report that config’s name and its error,
instead of incorrectly blaming an earlier override. We still fall back
to the original combined error behavior if no single config produces
the full-render failure.

How to verify it
See https://issues.redhat.com/browse/OCPBUGS-54692

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from sergiordlr May 13, 2025 14:57
@openshift-ci-robot
Copy link
Contributor

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-56119, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.19.0) matches configured target version for branch (4.19.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-54697 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-54697 targets the "4.20.0" version, which is one of the valid target versions: 4.20.0
  • bug has dependents

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

  • What I did
    Fixes: #OCPBUGS-56119
    Fix misleading error in generateAndValidateRenderedMachineConfig

When a full-render reconciliation fails, we now iterate through the
base MachineConfigs one by one—to
pinpoint the exact config that introduced the irreconcilable change.
If a partial render fails, we report that config’s name and its error,
instead of incorrectly blaming an earlier override. We still fall back
to the original combined error behavior if no single config produces
the full-render failure.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 13, 2025
@djoshy
Copy link
Contributor

djoshy commented May 13, 2025

/lgtm
/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label May 13, 2025
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 13, 2025
Copy link
Contributor

openshift-ci bot commented May 13, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, dkhater-redhat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [djoshy,dkhater-redhat]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dkhater-redhat dkhater-redhat changed the title OCPBUGS-56119: error from generateAndValidateRenderedMachineConfig function can be misleading [release-4.19] OCPBUGS-56119: error from generateAndValidateRenderedMachineConfig function can be misleading May 13, 2025
@dkhater-redhat
Copy link
Contributor Author

/cherry-pick release-4.18 release-4.17 release-4.16

@openshift-cherrypick-robot

@dkhater-redhat: once the present PR merges, I will cherry-pick it on top of release-4.18 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.18 release-4.17 release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ptalgulk01
Copy link

Pre-merge verification:
Verified using IPI based AWS cluster

launch 4.19,openshift/machine-config-operator#5033 aws

Applied the below MC which include the fips configuration

MC Example
 oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: luks-worker
spec:
  config:
    ignition:
      version: 3.4.0
    storage:
      filesystems:
        - device: /dev/mapper/root
          format: xfs
          label: root
          wipeFilesystem: true
      luks:
        - clevis:
            tpm2: true
          device: /dev/disk/by-partlabel/root
          label: luks-root
          name: root
          wipeVolume: true
EOF

Check the MCP degrade logs does not include 00-worker

  - lastTransitionTime: "2025-05-14T05:34:32Z"
    message: 'Failed to render configuration for pool worker: could not generate rendered
      MachineConfig: reconciliation failed between current config "rendered-worker-68b09b47b5a0d60933a1245c37892a78"
      and base config "luks-worker": ignition filesystems section contains changes'
    reason: ""
    status: "True"
    type: RenderDegraded

/label cherry-pick-approved

@openshift-ci openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label May 14, 2025
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD e9e9d5e and 2 for PR HEAD 2b07d56 in total

Copy link
Contributor

openshift-ci bot commented May 14, 2025

@dkhater-redhat: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-op-ocl 2b07d56 link false /test e2e-gcp-op-ocl
ci/prow/e2e-azure-ovn-upgrade-out-of-change 2b07d56 link false /test e2e-azure-ovn-upgrade-out-of-change

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 18f1c3d into openshift:release-4.19 May 14, 2025
19 of 21 checks passed
@openshift-ci-robot
Copy link
Contributor

@dkhater-redhat: Jira Issue OCPBUGS-56119: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-56119 has been moved to the MODIFIED state.

In response to this:

What I did
Fixes: #OCPBUGS-56119
Fix misleading error in generateAndValidateRenderedMachineConfig

When a full-render reconciliation fails, we now iterate through the
base MachineConfigs one by one—to
pinpoint the exact config that introduced the irreconcilable change.
If a partial render fails, we report that config’s name and its error,
instead of incorrectly blaming an earlier override. We still fall back
to the original combined error behavior if no single config produces
the full-render failure.

How to verify it
See https://issues.redhat.com/browse/OCPBUGS-54692

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-cherrypick-robot

@dkhater-redhat: new pull request created: #5049

In response to this:

/cherry-pick release-4.18 release-4.17 release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

Distgit: ose-machine-config-operator
This PR has been included in build ose-machine-config-operator-container-v4.19.0-202505141514.p0.g18f1c3d.assembly.stream.el9.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.