TestMachineSetReconciler is flaky

[sbueringer]

Which jobs are flaking?

At least pull-cluster-api-test-main

Which tests are flaking?

sigs.k8s.io/cluster-api/internal/controllers/machineset: TestMachineSetReconciler_syncReplicas_WithErrors/should_hold_off_on_sync_replicas_when_create_Infrastructure_of_machine_failed_

Since when has it been flaking?

Since we merged this unit test

Testgrid link

No response

Reason for failure (if possible)

The race detector found a data race: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cluster-api/11718/pull-cluster-api-test-main/1881643904957157376

Anything else we need to know?

Test was merged just yesterday

Label(s) to be applied

/kind flake
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[sbueringer]

/help

[k8s-ci-robot]

@sbueringer:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[sbueringer]

/triage accepted
/priority important-longterm

Probably we just have to use a separate scheme for TestMachineSetReconciler_syncReplicas_WithErrors

[Karthik-K-N]

I will work on it if no one started yet.

[enxebre]

/assign @Karthik-K-N

[Karthik-K-N]

Even after using -race flag not able to reproduce locally, Any tips?

[sbueringer]

Not sure, probably just not happening that often or only with CPU starvation. I think in general it's fine to just see if we can use a separate scheme for this unit test (+ then check via periodics in Prow over time if the flaky test goes away)

(you can check some other tests / places how we create a fake client with a scheme)

[sbueringer]

CI link: https://storage.googleapis.com/k8s-triage/index.html?pr=1&job=.*cluster-api.*main&test=should_hold_off_on_sync_replicas_when_create_Infrastructure_of_machine_failed_&xjob=.*-e2e-.*%7C.*-provider-.*

[Karthik-K-N]

Got the point, will do thanks.

[cprivitere]

The test this PR is about was added in PR: #11211