Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[windows] Nightly builds need more coverage #79020

Closed
weliveindetail opened this issue Jan 29, 2025 · 11 comments
Closed

[windows] Nightly builds need more coverage #79020

weliveindetail opened this issue Jan 29, 2025 · 11 comments

Comments

@weliveindetail
Copy link
Member

The infrastructure for running a stable subset of LLDB tests on Windows is in place on mainline. We enabled it for PR-testing with #78844, but https://ci-external.swift.org/job/swift-main-windows-toolchain/ uses a custom (hardcoded) invocation that I cannot access.

Bringing LLDB testing to swift-main-windows-toolchain seems important to monitor flakes. We filtered loads of flaky test in the bringup phase, but future changes may introduce new ones and it's critical to find them early. Otherwise they might clog the PR-testing bots.

Right now, it invokes swift\utils\build.ps1 directly and passes -Test swift,dispatch,xctest. We could either extend it to -Test swift,dispatch,xctest,lldb or drop the parameter entirely so that build.ps1 selects the default.

@weliveindetail
Copy link
Member Author

@shahmishal What do you think?

@weliveindetail
Copy link
Member Author

While we are here, we should also add -Summary to monitor build and tests times.

@weliveindetail
Copy link
Member Author

New test flake that popped up randomly in one of my own PRs: #79154 We should get better monitoring here.

@weliveindetail
Copy link
Member Author

@shahmishal Gentle ping. Is there anything I can do to help with this task?

@weliveindetail
Copy link
Member Author

@shahmishal Any chance to move on here this week?

It's hard to track flakes in PR-testing. And it's hard to tell how much it slows down the review process. Updating the mainline bots would help a lot.

@weliveindetail
Copy link
Member Author

@shahmishal ping

@weliveindetail
Copy link
Member Author

@shahmishal ping

@weliveindetail
Copy link
Member Author

@shahmishal ping

We also see more and more issues when building the Android SDKs. Might be useful to enable them (or at least one) in the nightly. @compnerd Can you confirm?

Summary: we need -Test swift,dispatch,xctest,lldb, -Summary and -Android (or at least -AndroidSDKs aarch64)

@weliveindetail weliveindetail changed the title [windows] Enable tests for LLDB in swift-main-windows-toolchain [windows] Nightly builds need more coverage Mar 11, 2025
@shahmishal
Copy link
Member

lldb added to the testing, -Summary and -Android flag now configured for Windows toolchain bots.

@shahmishal
Copy link
Member

@weliveindetail
Copy link
Member Author

Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants