-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: note musl based static builds, that work across linux distros + alpine ffi static & shared libs #416
Comments
Nice! We should update here also: https://docs.pact.io/docker |
So we have an issue with the pact_ffi shared library, for aarch64-musl
Error
Seems to be exclusive to aarch64, and the dynamic library only. amd64 tests are passing in pact-php as well using the shared library, and the static .a is being used and tested successfully in pact-js-core in this PR First reported by @tienvx over in pact-php alpine PR - Thanks Tien! |
So I've just built the aarch64 musl libs locally on my macbook, pact-reference/rust/pact_ffi/release-linux.sh Lines 82 to 111 in e44458a
Copied these over to pact-php
Green php tests. note have tried regenerating the |
## Rationale pact-reference has introduced musl and arm64 based ffi libraries for linux - pact-foundation/pact-reference#416 Tracking Issue - pact-foundation/roadmap#30 ## Issues Resolved fixes pact-foundation#498 fixes pact-foundation#496 fixes pact-foundation#500 fixes pact-foundation#374 fixes pact-foundation#387 ## Backwards Compatibility Linux glibc based hosts take precedence, so if any error occurs during musl detection. I do not anticipate breaking changes for users ## Implementation notes ### .NET notes - Docs - [Uses MSBuild Exec task](https://learn.microsoft.com/en-us/visualstudio/msbuild/exec-task?view=vs-2022) - MSBuild Blog Posts - [Cross-Platform Build Events in .NET Core using MSBuild](https://jeremybytes.blogspot.com/2020/05/cross-platform-build-events-in-net-core.html) - [MSBuild 101: Using the exit code from a command](https://www.creepingcoder.com/2020/06/01/msbuild-101-using-the-exit-code-from-a-command/) - Stack OverFlow - [Set PropertyGroup property to Exec output](https://stackoverflow.com/questions/76583824/set-propertygroup-property-to-exec-output) - .NET runtime musl detection code - https://github.com/dotnet/runtime/blob/a50ba0669353893ca8ade8568b0a7d210b5a425f/src/mono/llvm/llvm-init.proj\#L7 - https://github.com/dotnet/runtime/blob/a50ba0669353893ca8ade8568b0a7d210b5a425f/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.Unix.cs\#L78t ### Conditions for execution musl detection will run if - if linux - if /lib/ld-musl-(x86_64|aarch64).so.1 exists - if ldd bin/sh | grep musl is true (musl lib is loaded, rather than glibc) will continue on error, reverting back to glibc based libaries. ### Supported musl targets should work for multiple musl based distroes if - /lib/ld-musl-(x86_64|aarch64).so.1 exists - ldd is available (available by default in alpine images) Tested on Alpine ARM64 / AMD64. ## Caveats - [.NET does not run under QEMU](https://github.com/dotnet/core/blob/main/release-notes/8.0/supported-os.md#qemu) affecting the ability to test multi-arch from a single system - .NET restore can take a long time when running under containers. - [Workaround](NuGet/Home#13062 (comment)): Set `DOTNET_NUGET_SIGNATURE_VERIFICATION` to `false` ## Compatibility ### Operating System Due to using a shared native library instead of C# for the main Pact logic only certain OSs are supported: | OS | Arch | Support | | ------------ | ----------- | -------------------------------------------------------------------| | Windows | x86 | ❌ No | | Windows | x64 | ✔️ Yes | | Linux (libc) | x86 | ❌ No | | Linux (libc) | x64 | ✔️ Yes | | Linux (musl) | x64 | ✔️ Yes (Tier 2)* | | Linux (libc) | ARM | ✔️ Yes (Tier 3)* | | Linux (musl) | ARM | ✔️ Yes (Tier 3)* | | OSX | x64 | ✔️ Yes | | OSX | ARM (M1/M2) | ✔️ Yes | #### Support - Tier 1 - Established - Full CI/CD support. - Users should not encounter issues - Full reproducible examples running in CI, should be provided by users raising issues - If using musl targets, users should attempt the same test on a libc target (such as debian) - Tier 2 - Recently introduced - Full CI/CD support. - Users may encounter issues - Full reproducible examples running in CI, should be provided by users raising issues - If using musl targets, users should attempt the same test on a libc target (such as debian) - Tier 3 - Recently introduced, No/limited CI/CD support. - Users may encounter issues - Full reproducible examples which can be run by maintainers locally, should be provided by users raising issues
I think this was due to stripping of the symbols from the static archive, prior to generating the shared object file. It's possible to force the generation of the *.so at compile time with some rust flags, so that has been performed along with a smoke test to ensure its loading correctly in #432 |
Updated compat docs now added to pact_ffi readme https://github.com/pact-foundation/pact-reference/tree/master/rust/pact_ffi#compatibility
Updated PR for docs page here |
## Rationale pact-reference has introduced musl and arm64 based ffi libraries for linux - pact-foundation/pact-reference#416 Tracking Issue - pact-foundation/roadmap#30 ## Issues Resolved fixes pact-foundation#498 fixes pact-foundation#496 fixes pact-foundation#500 fixes pact-foundation#374 fixes pact-foundation#387 ## Backwards Compatibility Linux glibc based hosts take precedence, so if any error occurs during musl detection. I do not anticipate breaking changes for users ## Implementation notes ### .NET notes - Docs - [Uses MSBuild Exec task](https://learn.microsoft.com/en-us/visualstudio/msbuild/exec-task?view=vs-2022) - MSBuild Blog Posts - [Cross-Platform Build Events in .NET Core using MSBuild](https://jeremybytes.blogspot.com/2020/05/cross-platform-build-events-in-net-core.html) - [MSBuild 101: Using the exit code from a command](https://www.creepingcoder.com/2020/06/01/msbuild-101-using-the-exit-code-from-a-command/) - Stack OverFlow - [Set PropertyGroup property to Exec output](https://stackoverflow.com/questions/76583824/set-propertygroup-property-to-exec-output) - .NET runtime musl detection code - https://github.com/dotnet/runtime/blob/a50ba0669353893ca8ade8568b0a7d210b5a425f/src/mono/llvm/llvm-init.proj\#L7 - https://github.com/dotnet/runtime/blob/a50ba0669353893ca8ade8568b0a7d210b5a425f/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.Unix.cs\#L78t ### Conditions for execution musl detection will run if - if linux - if /lib/ld-musl-(x86_64|aarch64).so.1 exists - if ldd bin/sh | grep musl is true (musl lib is loaded, rather than glibc) will continue on error, reverting back to glibc based libaries. ### Supported musl targets should work for multiple musl based distroes if - /lib/ld-musl-(x86_64|aarch64).so.1 exists - ldd is available (available by default in alpine images) Tested on Alpine ARM64 / AMD64.
it would be good to document this on the readme, so users are aware that
*.a
& shared*.so
variants for musl, depending on the consuming client libraries requirements for ffi interopx86_64
.a
from 0.2.4aarch64
.a
from 0.4.15.so
from 0.4.17The text was updated successfully, but these errors were encountered: