Buildbot

Buildbot configuration

UV-CDAT uses Buildbot for continuous integration testing. To allow for multiple architectures and platforms, we use a custom setup of Buildbot for UV-CDAT's regression tests.

What is buildbot?

Buildbot is a tool which uses buildslaves to run various build configurations ("builders") on behalf of projects. The buildmaster watches for changes (typically in source repositories) and adds them to a queue for schedulers to hand off the buildslaves to run. These builders report their progress back to the buildmaster which uses "status" watchers to be able to do things such as report to IRC, show a webpage, etc.

The setup used here is that each machine describes a set of builders to build for each project on itself. These builders are built up using information from the project and slave using a "buildset" to describe the selection of features from the project to use. This allows the project configuration to describe what it really means to have a "python" build rather than each machine copying the same CMake variables all over the place. Instead, a buildset contains "python" and the project describes what that means.

UV-CDAT's buildbot setup

Before we get into the features of GitRobot, some background on how buildbot is set up could be useful. Builders are named such that they list

• the project name (uvcdat)
• the host machine (e.g. karma)
• the operating system (e.g. linux)
• the build type (release, debug)
• the build mode (lean, default, all)

Features indicate settings such as Qt GUI, etc. They are prefixed with a + and are sorted in alphabetical order. As an example, one of the builders on machine karma is:

uvcdat-karma-linux-debug-lean+gui

which indicates that it is a linux machine building UV-CDAT in Debug mode as a Lean build with GUI support.

If a pull request for branch meshfill_leak with the SHA-1 sum for HEAD commit as abbfec5a is despatched for testing to Buildbot, it reports a build on UV-CDAT's CDash page with the name:

abbfec5a-build122-[linux-debug-lean+gui]-meshfill_leak

Note that the additional text (build122) in the title denotes that this was the 122^nd build for buildbot.

For notes on adding a new build slave to the buildbot configuration, please refer to Add a Buildslave page.

GitRobot

Now that we are familiar with the Buildbot setup for UV-CDAT, let's dive into scheduling checks, builds and tests for changes.

UV-CDAT's custom Buildbot works in conjunction with GitRobot, a tangelo service that allows users to enter commands as comments on GitHub. In short, GitRobot provides a simple interface for scheduling tests with Buildbot. Not only does GitRobot schedule tests with Buildbot, it also customizes tests and their configurations providing the user with faster turn-around time for their branches.

Main GitRobot commands

The main commands available for users are:

Do: check

Checks the branch for inconsistencies in coding conventions, whitespace errors, etc. The "LLNL Robot" (@llnlbot) automatically checks the branch and adds a comment status based on the check results. These checks will be repeated automatically whenever the HEAD branch is changed.

Do: test

Schedules tests with buildbot. This command supports flags as described in Do: test flags section. The behavior with no flag is to build on all builders which are watching the project.

Do: merge

After a pull request has been reviewed and approved, authorized developers may add this comment to ask that the change be merged. By convention, refrain from issuing a request to merge if there are unresolved issues with the pull request arising from review comments.

If the merge succeeds, the topic will appear in the master branch and the pull request will be marked as merged automatically.

If the merge fails, Kitware Robot will add a comment describing the failure. In the case of a conflict, rebase the branch on to master.

Do: test flags

The Do: test command accepts the following arguments:

• --stop Tells buildbot that it should not schedule any more builds for the branch (in progress builds will continue) and should not trigger on branch updates. This is useful to save time if you know things have gone wrong (e.g. a builder has failed during configure or a typo triggered a build failure) and a quick fix is not in store. This frees up time so that other branches can be tested sooner.
• --clear Clear previous commands before adding this command
• --regex-include <arg> or -i <arg> Only build on builders matching <arg> (a Python regular expression)
• --regex-exclude <arg> or -e <arg> Excludes build on builders matching <arg> (a Python regular expression)

Examples

For testing on all slave machines, simply commenting with

Do: test

triggers build-and-test on all available build slaves.

If you find out it has errors only on OS X machines and push a fix for it,

Do: test --clear --regex-include osx

would clear previous commands and only test the new changes on OS X machines.

Finally, if you are left with errors just on the slave named macbot1, use

Do: test --clear --regex-include macbot1

To test all GUI builds not on the slave named karma, use

Do: test --regex-include \+gui --regex-exclude karma

NOTE: When using regex commands, make sure to escape special regular expression characters like + as shown in the example above.