Skip to content

Initial support for >=3.6 firmwares, toolchain v4.x #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: v4.x
Choose a base branch
from
Open

Initial support for >=3.6 firmwares, toolchain v4.x #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Toltec Build Toolchain

This branch contains the Docker image definitions for the `3.x` releases.\
This branch contains the Docker image definitions for the `4.x` releases.\
[See the main README file for more information →](https://github.com/toltec-dev/toolchain#readme)
89 changes: 41 additions & 48 deletions base/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@
ARG FROM
FROM $FROM

# Build libcap 2.32 targeting armhf
COPY patch/* /

# Build libcap 2.66 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Install build dependencies
&& apt-get update -y \
&& apt-get install -y --no-install-recommends \
git \
# Build libcap
&& cd /root \
&& git clone git://git.kernel.org/pub/scm/libs/libcap/libcap.git \
&& git clone --depth 1 --branch libcap-2.66 git://git.kernel.org/pub/scm/libs/libcap/libcap.git \
&& cd libcap \
&& git checkout 13227f9b2f2b2f222e8022e19bd46db6f6f898c6 \
&& sed -i "s/^BUILD_GPERF/#\0/" Make.Rules \
# Compile binaries that run on the build machine using normal GCC
&& sed -i "s/^\(BUILD_CC\) := \$(CC)/\1 := gcc/" Make.Rules \
&& sed -i "s/^\(BUILD_CC\) ?= \$(CC)/\1 := gcc/" Make.Rules \
&& make \
AR="${CROSS_COMPILE}ar" \
CC="${CROSS_COMPILE}gcc" \
Expand All @@ -33,7 +33,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log /var/log/apt

# Build util-linux 2.36.1 targeting armhf
# Build util-linux 2.37.4 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Install build dependencies
&& apt-get update -y \
Expand All @@ -46,17 +46,16 @@ RUN export DEBIAN_FRONTEND=noninteractive \
libtool \
# Build util-linux
&& cd /root \
&& git clone https://github.com/karelzak/util-linux.git \
&& git clone --depth 1 --branch v2.37.4 https://github.com/karelzak/util-linux.git \
&& cd util-linux \
&& git checkout 35c07c82be1ddc3b1c40f061b59008cac6405499 \
&& ./autogen.sh \
&& ./configure --host="$CHOST" \
&& make LDFLAGS="-Wl,-rpath-link,.libs" \
&& make install DESTDIR="$SYSROOT" \
&& cd .. \
# Clean up
&& rm -rf util-linux \
&& find "$SYSROOT" -type l,f -name "*.la" | xargs --no-run-if-empty rm \
&& find "$SYSROOT" -type l,f -name "*.la" -delete \
&& apt-get autoremove -y \
automake \
autopoint \
Expand All @@ -67,8 +66,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log /var/log/apt

# Build systemd 244 targeting armhf
COPY patch/*.patch /
# Build systemd 250 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Install build dependencies
&& apt-get update -y \
Expand All @@ -78,18 +76,12 @@ RUN export DEBIAN_FRONTEND=noninteractive \
gperf \
m4 \
rsync \
python3-pip \
&& pip3 install --no-cache-dir --break-system-packages jinja2 \
# Build systemd
&& cd /root \
&& git clone https://github.com/systemd/systemd.git \
&& git clone --depth 1 --branch v250.5 https://github.com/systemd/systemd.git \
&& cd systemd \
&& git checkout db9c5ae73e23d816e2df2a3e10a9a2a60b5b3ed7 \
# Fix build errors with GCC 10 and -O3
# (see <https://github.com/systemd/systemd/pull/15762>)
&& git apply /systemd-244-15762.patch \
# Fix Meson syntax and semantic changes in recent versions
# (see <https://github.com/systemd/systemd/pull/20633>)
&& git apply /systemd-244-20633.patch \
&& rm /systemd-244-*.patch \
&& ./configure \
--buildtype=release \
--cross-file="$CHOST" \
Expand All @@ -103,16 +95,18 @@ RUN export DEBIAN_FRONTEND=noninteractive \
&& cd .. \
# Clean up
&& rm -rf systemd \
# TODO: remove pip packages as well
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's required, they are pretty small (jinja2+yaml, I think)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed replying to this the first time. Yes, removing the packages is required as it's not part of what we intend to deliver by default, especially if we aren't delivering pip.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking that we can use multi-stage builds: one that cross-compiles all the stuff, then copy the headers/libs to the final one with pre-installed compilers without any extra deps. This should require less friction and layers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be appreciated, especially if there is caching we can do to avoid rebuilding if we already have a built version later.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, docker allows to cache multi-stage builds with buildx, that should be supported by GHA, will look into that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had issues with it not properly caching before, and there is also a limit to how much can be cached, which this would fill it up really quick, so you'd probably still want to clean up as much as possible to keep the cache size needed small.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better way to do this would be to move the build of these artifacts to a different process that stores them elsewhere with a version number, and then the docker image pulls the versioned artifact.

&& apt-get autoremove -y \
gettext \
git \
gperf \
m4 \
rsync \
python3-pip \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log /var/log/apt

# Build zlib 1.2.11 and libpng 1.6.37 targeting armhf
# Build zlib 1.2.11 and libpng 1.6.39 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Install build dependencies
&& apt-get update -y \
Expand All @@ -122,90 +116,85 @@ RUN export DEBIAN_FRONTEND=noninteractive \
libtool \
# Build static zlib
&& cd /root \
&& git clone https://github.com/madler/zlib \
&& git clone --depth 1 --branch v1.2.11 https://github.com/madler/zlib \
&& cd zlib \
&& git checkout cacf7f1d4e3d44d871b605da3b647f07d718623f \
&& CC="${CROSS_COMPILE}gcc" CFLAGS=-fPIC ./configure \
--static \
--prefix=/usr \
&& make \
&& DESTDIR="$SYSROOT" make install \
&& cd .. \
# Build dynamic libpng
&& git clone git://git.code.sf.net/p/libpng/code libpng \
&& git clone --depth 1 --branch v1.6.39 git://git.code.sf.net/p/libpng/code libpng \
&& cd libpng \
&& git checkout a40189cf881e9f0db80511c382292a5604c3c3d1 \
&& ./configure --prefix=/usr --host="$CHOST" \
&& make \
&& DESTDIR="$SYSROOT" make install \
&& cd .. \
# Clean up
&& rm -rf zlib libpng \
&& find "$SYSROOT" -type l,f -name "*.la" | xargs --no-run-if-empty rm \
&& find "$SYSROOT" -type l,f -name "*.la" -delete \
&& apt-get autoremove -y \
automake \
git \
libtool \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log /var/log/apt

# Build libevdev 1.9.1 targeting armhf
# Build libevdev 1.12.1 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Build libevdev
&& cd /root \
&& mkdir libevdev \
&& cd libevdev \
&& curl https://www.freedesktop.org/software/libevdev/libevdev-1.9.1.tar.xz -o libevdev.tar.xz \
&& echo "f5603c48c5afd76b14df7a5124e0a94a102f8da0d45826192325069d1bbc7acb libevdev.tar.xz" > sha256sums \
&& sha256sum -c sha256sums \
&& curl https://www.freedesktop.org/software/libevdev/libevdev-1.12.1.tar.xz -o libevdev.tar.xz \
&& echo "1dbba41bc516d3ca7abc0da5b862efe3ea8a7018fa6e9b97ce9d39401b22426c libevdev.tar.xz" | sha256sum -c \
&& tar --strip-components=1 -xf libevdev.tar.xz \
&& rm libevdev.tar.xz sha256sums \
&& rm libevdev.tar.xz \
&& ./configure --prefix=/usr --host="$CHOST" \
&& make \
&& DESTDIR="$SYSROOT" make install \
&& cd .. \
# Clean up
&& rm -rf libevdev \
&& find "$SYSROOT" -type l,f -name "*.la" | xargs --no-run-if-empty rm
&& find "$SYSROOT" -type l,f -name "*.la" -delete

# Build OpenSSL 1.1.1g targeting armhf
# Build OpenSSL 3.0.7 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Build OpenSSL
&& cd /root \
&& mkdir openssl \
&& cd openssl \
&& curl https://www.openssl.org/source/openssl-1.1.1g.tar.gz -o openssl.tar.gz \
&& echo "ddb04774f1e32f0c49751e21b67216ac87852ceb056b75209af2443400636d46 openssl.tar.gz" > sha256sums \
&& sha256sum -c sha256sums \
&& curl https://www.openssl.org/source/openssl-3.0.7.tar.gz -o openssl.tar.gz \
&& echo "83049d042a260e696f62406ac5c08bf706fd84383f945cf21bd61e9ed95c396e openssl.tar.gz" | sha256sum -c \
&& tar --strip-components=1 -xf openssl.tar.gz \
&& rm openssl.tar.gz sha256sums \
&& rm openssl.tar.gz \
&& ./Configure --prefix=/usr --cross-compile-prefix="$CROSS_COMPILE" linux-armv4 \
&& make \
&& make DESTDIR="$SYSROOT" install_sw \
&& cd .. \
# Clean up
&& rm -rf openssl

# Build libcurl 7.69.1 targeting armhf
# Build libcurl 7.82.0 targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Build libcurl
&& cd /root \
&& mkdir libcurl \
&& cd libcurl \
&& curl https://curl.se/download/curl-7.69.1.tar.xz -o curl.tar.xz \
&& echo "03c7d5e6697f7b7e40ada1b2256e565a555657398e6c1fcfa4cb251ccd819d4f curl.tar.xz" > sha256sums \
&& sha256sum -c sha256sums \
&& curl https://curl.se/download/curl-7.82.0.tar.xz -o curl.tar.xz \
&& echo "0aaa12d7bd04b0966254f2703ce80dd5c38dbbd76af0297d3d690cdce58a583c curl.tar.xz" | sha256sum -c \
&& tar --strip-components=1 -xf curl.tar.xz \
&& rm curl.tar.xz sha256sums \
&& rm curl.tar.xz \
&& ./configure --prefix=/usr --host="$CHOST" --with-openssl \
&& make \
&& DESTDIR="$SYSROOT" make install \
&& cd .. \
# Clean up
&& rm -rf libcurl \
&& find "$SYSROOT" -type l,f -name "*.la" | xargs --no-run-if-empty rm
&& find "$SYSROOT" -type l,f -name "*.la" -delete

# Build breakpad 0.1 (db1cda2653) targeting armhf
# Build breakpad 0.1 (216cea7bca) targeting armhf
RUN export DEBIAN_FRONTEND=noninteractive \
# Install build dependencies
&& apt-get update -y \
Expand All @@ -214,23 +203,25 @@ RUN export DEBIAN_FRONTEND=noninteractive \
zlib1g-dev \
libssl-dev \
# breakpad's build tools require python2 to be available as python on the path
&& git clone https://github.com/pyenv/pyenv.git .pyenv \
&& cd /root \
&& git clone --depth 1 https://github.com/pyenv/pyenv.git .pyenv \
&& export PYENV_ROOT="$(pwd)/.pyenv" \
&& export PATH="$PYENV_ROOT/bin:$PATH" \
&& eval "$(pyenv init --path)" \
&& pyenv install 2.7 \
&& pyenv global 2.7 \
# Fetch and activate depot_tools
&& cd /root \
&& git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git \
&& export PATH=$PATH:/root/depot_tools \
# Build static breakpad
&& cd /root \
&& mkdir breakpad \
&& cd breakpad \
&& fetch breakpad \
&& cd src \
&& git checkout db1cda26539c711c3da7ed4d410dfe8190e89b8f \
# fetch updates default.xml with seemingly useless diffs that break checkout later
&& git checkout HEAD -- default.xml \
&& git checkout 216cea7bca53fa441a3ee0d0f5fd339a3a894224 \
&& git apply < /breakpad-216cea7bca-fix-int-type.patch \
&& ./configure --prefix=/usr --host="$CHOST" \
&& make \
&& DESTDIR="$SYSROOT" make install \
Expand All @@ -249,3 +240,5 @@ RUN export DEBIAN_FRONTEND=noninteractive \
libssl-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log /var/log/apt

RUN rm /*.patch
18 changes: 18 additions & 0 deletions base/patch/breakpad-216cea7bca-fix-int-type.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
This patch fixes the issue with gcc complainig about following:
error: no matching function for call to ‘max(int, long int)’

Ref: https://github.com/pytorch/pytorch/issues/70392#issuecomment-1003488196

diff --git a/src/client/linux/handler/exception_handler.cc b/src/client/linux/handler/exception_handler.cc
index b895f6d7..dc70527b 100644
--- a/src/client/linux/handler/exception_handler.cc
+++ b/src/client/linux/handler/exception_handler.cc
@@ -138,7 +138,7 @@ void InstallAlternateStackLocked() {
// SIGSTKSZ may be too small to prevent the signal handlers from overrunning
// the alternative stack. Ensure that the size of the alternative stack is
// large enough.
- static const unsigned kSigStackSize = std::max(16384, SIGSTKSZ);
+ static const unsigned kSigStackSize = std::max(16384L, SIGSTKSZ);

// Only set an alternative stack if there isn't already one, or if the current
// one is too small.
69 changes: 0 additions & 69 deletions base/patch/systemd-244-15762.patch
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions base/patch/systemd-244-20633.patch
View file
Open in desktop

This file was deleted.

Loading