feat: add post-check on the params.env file for workbench images

[zdtsw]

Description

it is not clear in some cases, nbc image did not get properly swapped. and this is causing, nbc pod keeping trying to pull image which does not exist, e.g in disconnected env a quay.io image wont/shouldnt be there for use
often this is fixed by restart nbc deployment.
root cause still unclear for now.

this PR is only to add a post check and if it does not work then reconcile again.

• to prevent failure when components accidentally pull down wrong image
• check is on certain keywords, e.g new image pullspec but can be extend to any string

https://issues.redhat.com/browse/RHOAIENG-11368

How Has This Been Tested?

Screenshot or short clip

Merge criteria

• You have read the contributors guide.
• Commit messages are meaningful - have a clear and concise summary and detailed explanation of what was changed and why.
• Pull Request contains a description of the solution, a link to the JIRA issue, and to any dependent or related Pull Request.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from zdtsw. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ykaliuta]

Do you mean, that params.env does not get updated by ApplyParams() call?

Is it correct Jira?

[zdtsw]

Do you mean, that params.env does not get updated by ApplyParams() call?

Is it correct Jira?

i got the wrong number from the reporter and pasted the wrong one, corrected now

[zdtsw]

/test opendatahub-operator-e2e

[ykaliuta]

i got the wrong number from the reporter and pasted the wrong one, corrected now

Ok, read the problem. So, is it ApplyParams call does not change params.env (what the PR addresses IIUC) or it's not called at all? There are no comments in the Jira

[zdtsw]

Do you mean, that params.env does not get updated by ApplyParams() call?

Is it correct Jira?

it is hard to know what exactly happened at that point of time.
either the params.env did not get updated? or it gets updated but kustomize did not use it?
it is a very rare case, and it can be "auto fix" because component gets reconciled.
but if reconcile did not get triggered, then workbench (so far several cases only reported on workbench) pod is trying to pull image set as the default value in params.env

[openshift-ci]

@zdtsw: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/opendatahub-operator-e2e	dd1d00b	link	true	/test opendatahub-operator-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ykaliuta]

Should we close it?