Skip to content

OCPBUGS-56211: Boot Image Controller should not degrade when golden configmap is slow to update #5051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

djoshy
Copy link
Contributor

@djoshy djoshy commented May 14, 2025

- What I did
I removed the wait loop for the golden configmap stamp in the MSBIC. Instead, the controller will exit the sync and attempt to resync again when the confgimap is updated. This means that the MSBIC will no longer degrade if the master pool takes a while to update, which can be due to myriad of issues, and typically not boot image related.

- How to verify it
Existing e2es should pass. This should not cause any substantial changes in the MSBIC's actual update behavior. The MSBIC should no longer degrade if the golden configmap never gets stamped, but I'm fairly confident we weren't directly testing for this failure mode.

@openshift-ci-robot openshift-ci-robot added jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels May 14, 2025
@openshift-ci-robot
Copy link
Contributor

@djoshy: This pull request references Jira Issue OCPBUGS-56211, which is invalid:

  • expected the bug to target the "4.20.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did
I removed the wait loop for the golden configmap stamp in the MSBIC. Instead, the controller will exit the sync and attempt to resync again when the confgimap is updated. This means that the MSBIC will no longer degrade if the master pool takes a while to update, which can be due to myriad of issues, and typically not boot image related.

- How to verify it
Existing e2es should pass. This should not cause any substantial changes in the MSBIC's actual update behavior. The MSBIC should no longer degrade if the golden configmap never gets stamped, but I'm fairly confident we weren't directly testing for this failure mode.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label May 14, 2025
@djoshy
Copy link
Contributor Author

djoshy commented May 14, 2025

/jira refresh

@openshift-ci openshift-ci bot requested review from umohnani8 and yuqi-zhang May 14, 2025 19:17
@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 14, 2025
@openshift-ci-robot
Copy link
Contributor

@djoshy: This pull request references Jira Issue OCPBUGS-56211, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.0) matches configured target version for branch (4.20.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 14, 2025
@openshift-ci openshift-ci bot requested a review from sergiordlr May 14, 2025 19:17
Copy link
Contributor

openshift-ci bot commented May 14, 2025

@djoshy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-azure-ovn-upgrade-out-of-change b2976fe link false /test e2e-azure-ovn-upgrade-out-of-change
ci/prow/e2e-gcp-op-ocl b2976fe link false /test e2e-gcp-op-ocl

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link
Contributor

@yuqi-zhang yuqi-zhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Any issues that arise from this should be degrading elsewhere as we discussed, so I think there's no scenario where this failure would specifically be helpful for the user.

/hold

for pre-merge qe verification, if necessary

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 15, 2025
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2025
Copy link
Contributor

openshift-ci bot commented May 15, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@djoshy
Copy link
Contributor Author

djoshy commented May 16, 2025

Chatted on slack with @ptalgulk01 , Given that this is not easy to test, we will skip premerge QE. Going to run payloads for extra safety:

/payload-job periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-gcp-ovn-serial

Copy link
Contributor

openshift-ci bot commented May 16, 2025

@djoshy: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/818d3cc0-3243-11f0-9c19-beaf7395377b-0

@djoshy
Copy link
Contributor Author

djoshy commented May 16, 2025

/payload-job periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-serial periodic-ci-openshift-release-master-ci-4.20-e2e-gcp-ovn-serial

Copy link
Contributor

openshift-ci bot commented May 16, 2025

@djoshy: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.20-e2e-aws-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/96564840-3243-11f0-9f71-c8ea3ad31035-0

@djoshy
Copy link
Contributor Author

djoshy commented May 16, 2025

/payload-job periodic-ci-openshift-release-master-nightly-4.20-e2e-gcp-ovn-serial

Copy link
Contributor

openshift-ci bot commented May 16, 2025

@djoshy: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.20-e2e-gcp-ovn-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/d26000b0-3243-11f0-8fed-e908061c32cf-0

@djoshy
Copy link
Contributor Author

djoshy commented May 16, 2025

/cherry-pick release-4.19 release-4.18 release-4.17

@openshift-cherrypick-robot

@djoshy: once the present PR merges, I will cherry-pick it on top of release-4.19 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.19 release-4.18 release-4.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants