Skip to content

Tips for Building Packages

Jump to bottom
Mosè Giordano edited this page May 25, 2020 · 2 revisions

Note: this page has been moved to BinaryBuilder.jl documentation: Tips for Building Packages

Tips for Building Packages

BinaryBuilder provides a convenient environment to enable cross-platform building. But, many libraries have complicated build scripts that may need to be adapted to support all of the BinaryBuilder targets.

If you have additional tips, please submit a PR with suggestions.

Initiating different shell commands based on target

Sometimes, you need to adapt build scripts based on the target platform. This can be done within the shell script. Here is an example from staticfloat/OpenBLASBuilder:

# Set BINARY=32 on i686 platforms and armv7l
if [[ ${nbits} == 32 ]]; then
    flags="${flags} BINARY=32"
fi

Here are other examples of scripts with target-specific checks:

  • XZ - Custom windows installation
  • Sundials - 32-bit check

It is also possible to run quite different scripts for each target by running different build scripts for different sets of targets. Here is an example where windows builds are separated from other targets:

Autoconfigure builds

Autoconfigure builds are generally quite straightforward. Here is a typical approach:

./configure --prefix=$prefix --host=${target}
make -j${nproc}
make install

Here are examples of autoconfigure build scripts:

CMake builds

For CMake, the wizard will suggest a template for running CMake. Typically, this will look like:

cmake -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} -DCMAKE_BUILD_TYPE=Release

The toolchain file sets up several CMake environment variables for better cross-platform support, such as CMAKE_SYSROOT, CMAKE_C_COMPILER, etc... Examples of builds that include CMake parts include:

  • JpegTurbo

  • Sundials

    • Needs -DSUNDIALS_INDEX_TYPE=int32_t on 32-bit targets (Sundials-specific way to specify integer size)
    • Needs to copy *.dll files from destdir/lib to destdir/bin for windows; this also removes symlinks by using cp -L
    • Needs -DCMAKE_FIND_ROOT_PATH="$WORKSPACE/destdir", so CMake's find_library can find libraries from KLU

Meson builds

BinaryBuilder supports also building with Meson. Since this is going to be a cross-compilation, you have to specify a Meson cross file:

meson --cross-file="${MESON_TARGET_TOOLCHAIN}"

After configuring the project with meson, you can then build and install it with

ninja -j${nproc}
ninja install

The wizard automatically suggests using Meson if the meson.build file is present.

Examples of builds performed with Meson include:

  • gdk-pixbuf

    • Here meson uses platform-dependent options

  • libepoxy

    • This script modifies c_args in the Meson cross file in order to add an include directory

  • xkbcommon

Builds with binary dependencies

A build script can depend on binaries generated by another builder. A builder specifies dependencies in the form of previously-built JLL packages:

# Dependencies of Xorg_xkbcomp
dependencies = [
    Dependency("Xorg_libxkbfile_jll"),
    BuildDependency("Xorg_util_macros_jll)",
]
  • Dependency specify a JLL package that is necessary to build and load the current builder;
  • BuildDependency is a JLL package necessary only to build the current package, but not to load it. This dependency will not be added to the list of the dependencies of the generated JLL package.

In the wizard, dependencies can be specified with the prompt: Do you require any (binary) dependencies? [y/N].

Examples of builders that depend on other binaries include:

  • ImageMagick depends on Zlib, libpng, JpegTurbo and Libtiff.

Editing files in the wizard

In the wizard, the vim editor is available for editing files. But, it doesn't leave any record in the build script. One generally needs to provide patch files or use something like sed. If a file needs patching, we suggest using git to add the entire worktree to a new repo, make the changes you need, then use git diff -p to output a patch that can be included alongside your build recipe.

You can include local files like patches very easily by placing them within a bundled/patches nested directory, and then providing "./bundled" as one of the sources for your build. See, for example, OpenBLAS.

Automatic environment variables

The following environment variables are automatically set in the build environment and should be used to build the project. Occasionally, you may need to tweak them (e.g., when Using GCC on macOS and FreeBSD).

  • CC: the C cross compiler
  • CXX: the C++ cross compiler
  • FC: the Fortran cross compiler
  • OBJC: the Objective-C cross compiler

The above variables point to utilities for the target environment. To reference the utilities for the host environment either prepend HOST or append _HOST. For example, HOSTCC and CC_HOST point to the native C compiler.

These are other environment variables that you may occasionally need to set during a build

  • CFLAGS: options for the C compiler
  • CXXFLAGS: options for the C++ compiler
  • CPPFLAGS: options for the C pre-processor
  • LDFLAGS: options for the linker
  • PKG_CONFIG_PATH: a colon-separated list of directories to search for .pc files
  • PKG_CONFIG_SYSROOT_DIR: modifies -I and -L to use the directories located in target sysroot

The following variables are useful to control the build script over different target systems, but are not intended to be modified by the users:

  • prefix: the path to the top-directory of where all the products should be installed. This will be the top-directory of the generated tarball
  • libdir: the path to the directory where the shared libraries should be installed. This is ${prefix}/bin when building for Windows, ${prefix}/lib for all other platforms
  • bindir: the path to the directory where the executables should be installed. This is equivalent to ${prefix}/bin
  • target: the target platform
  • bb_full_target: the full target platform, containing things like libstdc++ string ABI platform tags, and libgfortran version
  • nproc: the number of processors of the host machine, useful for parallel building (e.g., make -j${nproc})
  • nbits: number of bits of the target architecture (usually it is either 32 or 64)
  • proc_family: target processor family (e.g., "intel", "power", or "arm")
  • dlext: extension of the shared library on the target system. It is "dll" for Windows, "dylib" for macOS, and "so" for the other Unix systems
  • SRC_NAME: name of the project being built

Using GCC on macOS and FreeBSD

For these target systems Clang is the default compiler, however some programs may not be compatible with Clang.

For programs built with CMake (see the CMake build section) you can use the GCC toolchain file that is in ${CMAKE_TARGET_TOOLCHAIN%.*}_gcc.cmake.

If the project that you want to build uses the GNU Build System (also knows as the Autotools), there isn't an automatic switch to use GCC, but you have to set the appropriate variables. For example, this setting can be used to build most C/C++ programs with GCC for FreeBSD and macOS:

if [[ "${target}" == *-freebsd* ]] || [[ "${target}" == *-apple-* ]]; then
    CC=gcc
    CXX=g++
fi

Installing the license file

Generated tarballs should come with the license of the library that you want to install. If at the end of a successful build there is only one directory inside ${WORKSPACE}/srcdir, BinaryBuilder will look into it for files with typical names for license (like LICENSE, COPYRIGHT, etc... with some combinations of extensions) and automatically install them to ${prefix}/share/licenses/${SRC_NAME}/. If in the final tarball there are no files in this directory a warning will be issued, to remind you to provide a license file.

If the license file is not automatically installed (for example because there is more than one directory in ${WORKSPACE}/srcdir or because the file name doesn't match the expected pattern) you have to manually install the file. In the build script you can use the install_license command. See the Utilities in the build environment section below.

Utilities in the build environment

In addition to the standard Unix tools, in the build environment there are some extra commands provided by BinaryBuilder. Here is a list of some of these commands:

  • atomic_patch: utility to apply patches. It is similar to the standard patch, but it fails gracefully when a patch cannot be applied: 
    atomic_patch -p1 /path/to/file.patch
  • install_license: utility to install a file to ${prefix}/share/licenses/${SRC_NAME}: 
    install_license ${WORKSPACE}/srcdir/THIS_IS_THE_LICENSE.md
  • update_configure_scripts: utility to update autoconfigure scripts. Sometimes libraries come with out-of-date autoconfigure scripts (e.g., old configure.guess can't recognise aarch64 platforms of systems using Musl C library). Just run 
    update_configure_scripts
    to get a newer version
Clone this wiki locally