Windows Mailpile Package Build

Windows Mailpile Package Build

Mailpile uses the WIX toolset to generate an MSI package for windows. To do so, it has to gather and prepare Mailpile's dependencies for packaging. A small python framework aptly named provide addresses those needs and serves as a scripting glue to automate generating a package. This page describes how to invoke, configure, and maintain the provide framework.

There are READMEs in the code base that describe how various components work, and detailed help screens for the tools in use. This page aims to stitch them all together at a high level.

Quick-Start

If you're looking to create a regular or signed package, look no further! These two sections describe how to go from zero to building packages.

Generating a Plain Package

Generating a package requires python (either 2.7 or 3.x), git, and a checkout of mailpile(~1.0.0rc3 or later). The build script is very configurable in effort to be portable/reproducible. Assuming git is on your PATH, building is as simple as changing directory to where you'd like the output, and invoking the build script:

python.exe path\to\mailpile\packages\windows-wix\provide -i path\to\mailpile\packages\windows-wix\provide-example-fork.json

If git isn't on your path, you can specify it's location:

python.exe path\to\mailpile\packages\windows-wix\provide ^
 -i path\to\mailpile\packages\windows-wix\provide-example-fork.json ^
 --configure-git=path\to\git.exe

It will download, process, and package dependencies, preparing a "package" directory in your current directory. You will need to manually grant privileges to unpack win4gpg when prompted. The directory will contain several files:

• mailpile-<version>-<lang>.msi the created mailpile package.
• mailpile-<version>-<lang>.wixpdb debugging symbols for the package.
• mailpile.uuid.json the UUIDs and sha1 hashes for ALL packaged files. This should be committed as part of the release process, updating/replacing mailpile\packages\windows-wix\package_uuid_db.json, so that identical files share a single UUID across package versions.
• mailpile.package.json the inflated template for mailpile\packages\windows-wix\package.py.
• mailpile.wxs the wix configuration script generated by provide.
• mailpile.wixobj the wixobj file made from the mailpile.wxs configuration.

The first should be sufficient to install mailpile. If making a release, it is recommended to keep the remainder for troubleshooting/archival purposes.

Generating a signed package

The provide script supports signing all PDE package content if signtool.exe and a code signing certificate is available. To secure signtool.exe, install the windows SDK(if only installing the code-signing checkbox, the install size is only a few MB). 'providewill attempt to find the most recent version ofsigntool.exeinstalled (the SDK version bumps very regularly)--you can check if it's found it by invokingpython.exe mailpile\packages\windows-wix\provide -hand looking for the default value of--config_signtool: If it is not simply signtool.exe, providewas able to find a version of the windows SDK. If need you can manually specifysigntool.exe` on the command line:

python.exe path\to\mailpile\packages\windows-wix\provide ^
 -i path\to\mailpile\packages\windows-wix\provide-example-fork.json ^
 --config_signtool="C:\Program Files(x86)\Windows Kits\10\bin\10.0.17134.0\x86\signtool.exe"

To actually sign the build, you must provide the path to the code signing certificate and password for the certificate:

python.exe path\to\mailpile\packages\windows-wix\provide ^
 -i path\to\mailpile\packages\windows-wix\provide-example-fork.json ^
 --config_signing_key=path\to\code_signing_key.p12 ^
 --config_signing_passwd=a_very_secure_passwd

The build proceeds exactly like a regular build, except that all PDE (exe, dll, msi) content is signed.

Maintaining Dependencies

Provide works by fetching and caching dependencies, processing them into a portable install state, and packaging them. From time to time it will be necessary to update or modify external dependencies for the build script. Dependency sources are kept in a simple JSON file: mailpile\packages\windows-wix\resources.json. They can be updated as needed: Either directly edit the JSON(providing URLs and SHA1s), or use the helper program mailpile\packages\windows-wix\provide\cache.py to calculate the SHA1s for you:

python.exe mailpile\packages\windows-wix\provide\cache.py ^
 -c mailpile\packages\windows-wix\download_cache ^
 -r mailpile\packages\windows-wix\resource.json ^
 -i "[\"resource_hacker\", \"https://www.mailpile.is/files/windows-deps/20180901/resource_hacker.zip\", \"optionally replace the previous comment text with this text\"]"

Resources are identified by a common name throughout the build scripts to decouple build script uses from remote sources. Currently mailpile uses the following sources as part of the build:

• CPython 2.7: provide downloads the known-good version, configures it with required packages, strips unnecessary additions, then either partially(fork build) or entirely(bootstrap build) uses it for the rest of the build.
• Tor: Bundled as a dependency.
• Win4gpg: provides relatively recent GPG infrastructure. Portable version used--see their documentation about "mkportable".
• OpenSSL: Bundled as a dependency.
• Resource Hacker: Used to make mailpile-branded python and wpython executable, not shipped in package.
• Mailpile and submodules: Checked-out or copied and reset to a specific commit(default most recent on current branch).

Customizing the Build

The provide script is highly configurable. In part, this is to facilitate portable/reproducible builds. It also facilitates maintainability and customization points by allowing various configuration points to be re-targeted at a very high level. It is also designed to safely decouple working state from package state.

Configuration file

At it's core, provide evaluates a declarative configuration JSON, building a one-off context in the process. The JSON key-value pairs are as described by python.exe provide -h. Configuration is evaluated lazily from one of two entry points:

• export: Export accepts a dictionary of build targets to export(copy) locations. It constructs each specified target and copies it to the specified location. The standard configuration for export is {"package": ".\\package\"}: construct the msi package product directory and copy it to the directory named package. Multiple targets can be specified to help debug packaging issues.
• bootstrap: Bootstrap accepts a configuration to evaluate within the build context. It allows you to ensure the build script version exactly matches the content being built. Bootstrap sets up python and a mailpile checkout/clean copy, then invokes the version of provide within that checkout with the configuration specified. A few items are carried over: log level, cache location, git path, etc.; however it is necessary to nest an export configuration within bootstrap. (Yes, multiple bootstrap recursion is possible, though not at all recommended.

Of the two, bootstrap should be preferred for reproducible builds, and export should be preferred for debugging.

Specifying an alternate branch or repository

By default, provide copies the current repository checkout, cleanly resets it, then performs a build. This nicely avoids re-checking out submodules etc. that might otherwise slow down a build, and is generally what people want most of the time. You can ask provide to build a specific branch/checkout from a specific repository at configuration time. In a configuration file, add the following to the root configuration object:

  "mailpile": {
    "commit": "<branch name or commit hash>",
    "repo": "<url to a mailpile git repository>"
  }

Or on the command line, add the following option:

--config_mailpile="{\"commit\": \"<branch name or commit hash>\", \"repo\": \"<url to a mailpile git repository>\"}"

See the example provide configuration JSONs in mailpile/packages/windows-wix.

Build Scripts

Targets are provided by as a modular/extensible scripting framework. See mailpile/packages/windows-wix/README.md.