Pre merge builds of GitHub pull requests

When a pull request gets created or updated a build will be triggered on our Jenkins CI server. The build results will be visible on the public Jenkins; the link is visible in the comment that Jenkins adds to the pull request after the build is finished. The builds can also be accessed on our internal Jenkins where they might give some more details.

Because we're building on both Windows and Linux the setup behind the scenes is slightly complicated. Unfortunately the published build results don't show all details, so it might be worth to look at the console output reachable from the linked build page (see below).

At the time of this writing only pull requests on the BloomDesktop repo have pre-merge builds set up.

Controlling builds

A pull request created or updated by a member of the Bloom GitHub group/organisation will automatically trigger a build. For other contributors a member of the team needs to kick off a build by adding a comment to the pull request:

• "ok to test" to accept this pull request for building/testing

• "test this please" for a one time test run

• "add to whitelist" to add the author to the whitelist

If the build fails for other various reasons you can rebuild:

• "retest this please" to start a new build

If a pull request of a team member should not trigger a build, the phrase "skip ci" can be included in the pull request message.

What happens

The following illustration illustrates what happens after a pull request gets created or updated:

A new or updated pull request triggers the job GitHub-Bloom-Wrapper-debug on Jenkins which kicks off a Windows (GitHub-Bloom-Win32-PR-debug) and a Linux build (GitHub-Bloom-Linux-any-PR-debug).

After these are finished the unit tests are run by separate jobs: GitHub-Bloom-Linux-any-PR-debug-Tests and Bloom-Win32-default-debug-Tests.

Finally the wrapper job collects the output and publishes the results on our public jenkins.

Troubleshooting failing builds

If a build fails you can follow the link to the build results on our public jenkins server. Unfortunately the overview page you'll see there isn't entirely reliable: if a build fails because of compile failures you might still see failing tests on the overview page which misleads into thinking that the build failed because of failing tests.

(The reason is that we have a single trigger job that triggers both Windows and Linux builds in order to get a single status on the GitHub PR. The consolidation of the build results has some limitations...)

I recommend to look at the Console Output of the linked trigger job (GitHub-Bloom-Wrapper-debug). This shows the triggered builds and the build status of the individual builds, e.g.

...
[workspace] $ /bin/sh -xe /tmp/hudson1343948848567641155.sh
+ echo -n jenkins-GitHub-Bloom-Wrapper-debug-229
Waiting for the completion of GitHub-Bloom-Linux-any-PR-debug
GitHub-Bloom-Linux-any-PR-debug #222 completed. Result was FAILURE
Waiting for the completion of GitHub-Bloom-Win32-PR-debug
GitHub-Bloom-Win32-PR-debug #221 completed. Result was SUCCESS
Waiting for the completion of GitHub-Bloom-Linux-any-PR--JSTests
GitHub-Bloom-Linux-any-PR--JSTests #41 completed. Result was UNSTABLE
Build step 'Trigger/call builds on other projects' marked build as failure
...

If you follow the link to the failed job and look at the console output of that you'll be able to see why the build failed. UNSTABLE means that there are test failures, FAILURE means that the build itself failed, i.e. compiling of the code failed or some other problem occured.