[Windows] Run swift-format tests from build.ps1

[ahoppen]

The basic idea is that we build all libraries for the executable that will be included in the toolchain using CMake. swift-format then has a mode in its Package manifest that allows it to build just the test and test support targets, requiring all search paths to find those libraries to be passed in. We use that to only build swift-format's test using SwiftPM and re-use all the libraries that were already built using CMake.

Building the test targets and running the tests takes about 1 minute locally on my machine.

[ahoppen]

https://github.com/swiftlang/swift-format

@swift-ci Please build toolchain Windows

[ahoppen]

swiftlang/swift-format#841

@swift-ci Please build toolchain Windows

[ahoppen]

swiftlang/swift-format#841
swiftlang/swift-installer-scripts#343

@swift-ci Please build toolchain Windows

[compnerd]

Please provide a before and after comparison for the file sizes. This was a HUGE regression the last time I checked. It was about 50MiB larger.

[compnerd]

(1) it means we don't need to build SwiftFormat twice
(2) we test the same build that is also included in the toolchain

Sure

(3) it allows us to remove the CMake build of swift-format.

We cannot do this yet. SPM does not properly link the binary, so there are still issues with it and until that is fixed, I don't think that we should be shipping the SPM built version.

[ahoppen]

Please provide a before and after comparison for the file sizes. This was a HUGE regression the last time I checked. It was about 50MiB larger.

Sorry, I meant to include these but then forgot. For a local toolchain build I get

• In a 6.0.0 development toolchain I downloaded and have installed in AppData\Local: swift-format.exe with 187KB + SwiftFormat.dll with 1,894KB = 2,081KB
• In the toolchain I downloaded from this CI run swift-format.exe is 2,146 KB
• With the old CMake-based build: swift-format.exe with 763KB + SwiftFormat.dll with 7.974KB = 8,737KB
• With the new SwiftPM-based build: swift-format.exe with 8,326KB

I don’t know where the difference between my local toolchain build and the toolchain downloaded from CI comes from but the gist is that sizes are very comparable (65KB increase in downloaded toolchains size, 411KB size reduction in local builds).

Note that we are not rebuilding and statically linking all the dependencies here. We are only building swift-format and dynamically linking against the already built versions of swift-syntax, swift-markdown etc.

We cannot do this yet. SPM does not properly link the binary, so there are still issues with it and until that is fixed, I don't think that we should be shipping the SPM built version.

Can you clarify what you mean by SwiftPM not linking correctly? From what I’ve tested so far swift-format.exe built using SwiftPM launches just fine.

[ahoppen]

swiftlang/swift-format#841
swiftlang/swift-installer-scripts#343

@swift-ci Please build toolchain Windows

[compnerd]

Note that we are not rebuilding and statically linking all the dependencies here. We are only building swift-format and dynamically linking against the already built versions of swift-syntax, swift-markdown etc.

This is ... unexpected. How do we accomplish this? I'd love to see this adopted in SPM as well!

Can you clarify what you mean by SwiftPM not linking correctly? From what I’ve tested so far swift-format.exe built using SwiftPM launches just fine.

Sure - the linker spew that you get when building is due to the binary being miscompiled but the linker workaround it. This is something that @dschaefer2 is planning on looking into and I think that we should wait for that.

[dschaefer2]

Can you clarify what you mean by SwiftPM not linking correctly? From what I’ve tested so far swift-format.exe built using SwiftPM launches just fine.

Sure - the linker spew that you get when building is due to the binary being miscompiled but the linker workaround it. This is something that @dschaefer2 is planning on looking into and I think that we should wait for that.

Yup, working on that right now. Almost have clean linking happening :).

[ahoppen]

This is ... unexpected. How do we accomplish this? I'd love to see this adopted in SPM as well!

I am manually specifying all the search paths with which SwiftPM can find the dynamic libraries built by the previous build stages and explicitly remove all explicit dependencies from swift-format’s package manifest through an environment variable.

https://github.com/swiftlang/swift/pull/76900/files#diff-c861caa2fcc08744108e542ca836b07e342f537566e8c5f7a21a0e9125331ed9R2373-R2389

[ahoppen]

I pivoted the approach around a little. Let’s decouple the discussion about how to build the swift-format executable from running tests.

I updated this PR to rely on CMake building everything needed for the toolchain. We then only build the test and test support targets in swift-format using SwiftPM with various search paths to make SwiftPM pick up the pre-built modules and libraries.

[ahoppen]

#76900

@swift-ci Please test Windows

[ahoppen]

swiftlang/swift-format#851

@swift-ci Please test Windows

[ahoppen]

swiftlang/swift-format#851

@swift-ci Please smoke test macOS

[ahoppen]

swiftlang/swift-format#851

@swift-ci Please smoke test Linux

[ahoppen]

Merging this so we get test coverage for swift-format on Windows and because this has no risk to the toolchain build because it doesn’t change how swift-format.exe is built. Happy to address any further feedback in a follow-up PR.