Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip flaky performance test on Windows #940

Conversation

weliveindetail
Copy link
Member

Let's skip testWhitespaceLinterPerformance on Windows until someone with more linter performance insights can craft a real fix for #939

Copy link
Contributor

@bnbarham bnbarham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @weliveindetail, was just about to put this up. These run in a VM, though the 200% variance is still odd to me. @allevato was this intended to run in CI anyway, or was it just for local testing?

@bnbarham bnbarham merged commit a5b5166 into swiftlang:main Feb 19, 2025
19 checks passed
@allevato
Copy link
Member

@allevato was this intended to run in CI anyway, or was it just for local testing?

Aside from being flaky, I can't think of a reason that we wouldn't want them to run in CI. But I'm not opposed to disabling them on platforms where it's currently a problem.

I can't think of anything Windows-specific that would cause the results to have such high variance, but I'm also not a frequent Windows user.

@bnbarham
Copy link
Contributor

Aside from being flaky, I can't think of a reason that we wouldn't want them to run in CI

It's really only testing for variance within the run in CI (unless I'm missing something?) and it seems unlikely that should ever occur outside of CI-VM-flakiness.

@allevato
Copy link
Member

Yeah, that's fair. I imagine there's an environment variable that we can check inside the test to decide whether to run it or not? Or, alternatively, check it inside the package manifest and not create that test target when it builds for CI?

@bnbarham
Copy link
Contributor

Yeah, we often use SWIFTCI_USE_LOCAL_DEPS for that.

Or, alternatively, check it inside the package manifest and not create that test target when it builds for CI?

SGTM

@weliveindetail weliveindetail deleted the windows-skip-flaky-testWhitespaceLinterPerformance branch February 20, 2025 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants