-
Notifications
You must be signed in to change notification settings - Fork 240
feat: add support to x64 systems using musl #632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Hey @xerial! This is a pull request that gives compatibility with systems using musl. Can you please take a look at this? Thanks! |
@samyuh It seems building a native library for Alipine is failing on CI. https://github.com/xerial/snappy-java/actions/runs/12839320244/job/36630155639?pr=632 |
Hey @xerial thanks for the feedback! I will try to make time to look into this later today :) |
Pipelines should pass now, I just renamed one file and forgot to change it on |
Hey @xerial |
Problem
When importing a data source of a file that was previously compressed using the snappy compression algorithm, the compression fails with the following exception:
This problem occurs since the dockerimage does not containg the
ld-linux-x86-64.so.2
lib, which is from glibc. Alpine images use musl making this incompatible.Having this dependency being linked dynamically can mess things up. Ideally, we would like to have snappy being compiled against a MUSL image.
Snappy removed pure-java mode 1.1.9.0 (https://github.com/xerial/snappy-java/pull/381/files), which could be used to run snappy in non-supported systems with the use of one flag. We could force an older version but it was removed due to some problems (such as data corruption) and older versions than 1.1.9.0 contain critical CVEs (https://mvnrepository.com/artifact/org.xerial.snappy/snappy-java).
Workaround
Alpine images also have some packages that allow us to run binaries precompiled against glibc libraries (they don't allow us to compile against it), such as gcompat. In our case,
libc6-compat
provides/lib64/ld-linux-x86-64.so.2
compatible lib. Turns out that/lib64/ld-linux-x86-64.so.2
is a symlink to/lib/libc.musl-x86_64.so.1
. Adding a/lib/ld-linux-x86-64.so.2
linked to/lib/ld-musl-x86_64.so.1
solves the issue since that is the place snappy is searching for the dependency.This is the same solution employed by https://stackoverflow.com/a/55568352.
Fix
The library now includes native binaries compiled specifically for musl environments, eliminating the need for workarounds like installing
gcompat
orlibc6-compat
packages in Alpine containers.This improvement means:
Closes #616
Closes #239