BFF v2.8

(These release notes are copied from our 5 October 2016 blog post announcing the release of BFF 2.8)

Announcing the release of the CERT Basic Fuzzing Framework Version 2.8 (BFF 2.8). It's been about three years since we released BFF 2.7. In this post, I highlight some of the changes we've made.

---

Release notes for versions prior to 2.8 can be found here

---

Your FOE Is Now Your BFF

To help reduce confusion over our fuzzing tools, the CERT Failure Observation Engine (FOE) is now known as BFF for Windows. For the past few versions, we have been converging the code bases for BFF and FOE into a unified architecture. While a few platform-specific differences remain, the name change reflects the fact that they are now essentially the same product with multi-platform support.

For clarity in this post, I'll refer to BFF for if we're talking about something platform-specific; otherwise I'll refer to BFF for features that are supported across platforms.

Significant Changes

BFF 2.8 has undergone a lot of changes since 2.7 was last released. Here is an overview of some of the bigger changes.

Configurable Mutators (New to Linux and OSX)

BFF for Linux and OSX now use the same configurable mutators as BFF for Windows. Prior to this release, BFF supported only bitwise mutation because it relied on zzuf for fuzzing and crash detection. FOE, on the other hand, had configurable mutators from day one, but was only available for Windows. With BFF 2.8, all platforms now default to using the bytemut mutator, which we have found to be more effective at searching the input space for crashing test cases.

BFF still uses zzuf on Linux and OSX for crash detection, but all mutation is now done directly in BFF's python code.

Verify Mode (New to Linux and OSX)

Having configurable mutators permits us to have a null mutator that does not modify the input files at all. As a result, Linux and OSX now support verify mode, another feature previously available only on FOE on Windows.

Verify mode can be useful in a few situations, including the following:

1. Say you've previously run BFF against a piece of software and found crashing test cases. At some point, if a newer version of the software becomes available, you might want to check to see which of the test cases are now fixed.
2. If you've got crashing test cases from another fuzzer, like American Fuzzy Lop (afl) or Peach Fuzzer, you could triage them using verify mode to run each test case through BFF's test case analysis pipeline to collect debugger output, exploitability estimates, core dumps, valgrind output, etc.

To use BFF in verify mode, do the following:

• Take the old crashing test cases and use them as seed files. This approach can be accomplished using tools/copycrashers.py.
• Run them through BFF in verify mode. Remember to configure the campaign to use the new version of the program you want to check them against.
• Look at the results directory to see which test cases are still a problem.

Drillresults on Every New Crash (All Platforms)

Drillresults was originally included with FOE 2.0 for Windows as a standalone script that you could run to identify easily exploitable vulnerabilities from a fuzzing campaign's results after the fact. Later we added it to BFF for Linux and OSX, but it remained as a standalone script.

In BFF 2.8, drillresults is now run automatically on each crash as part of BFF's post-crash analysis pipeline. Each crashing testcase directory now contains a file with the .drillresults extension containing that information.

Architecture (All Platforms)

Under the hood, we've done quite a bit of refactoring to eliminate redundancies across the Linux and Windows codebases. The overall BFF architecture is now platform agnostic, with OS-specific code implemented in separate modules and subclasses where necessary. This consolidation allows us to more easily add new features across all the platforms that BFF supports without having to duplicate any more code than necessary.

Support for Recent OSX Versions

BFF for OSX should work on Mavericks, Yosemite, El Capitan, and Sierra.

Added !analyze Output (Windows)

BFF for Windows collects !analyze -v output (via CDB) for each unique crashing test case. Microsoft published more information about !analyze in their debugger reference materials.

Updated !exploitable (Windows)

BFF for Windows now uses Microsoft's !exploitable version 1.6.

Simplified Configuration (All Platforms)

The BFF configuration file, bff.yaml, was simplified to make configuring fuzzing campaigns easier.

Self-Update Capability (All Platforms)

BFF includes a utility called updatebff.py in the tools directory. Simply run tools/updatebff.py (or on Windows, tools\updatebff.py) to install the latest certfuzz code from GitHub.

Contributing and GitHub Availability

In early 2014 we converted our development process from svn to git, which also allows us to start pushing the work-in-progress code to GitHub. While our day-to-day development still happens in house, having the code available on GitHub allows us to work more directly with, and be more responsive to, outside contributors. It also gives BFF users a place to report bugs or make feature requests.

You can find the code on GitHub. You can also report a bug or request a feature on GitHub.

Download BFF

BFF 2.8 is available for download on our website.

Change Log for BFF 2.8

Bug

• [BFF-178] - Updated config file should trump cached config
• [BFF-262] - FOE null runner heisenbug / unable to get md5 issues
• [BFF-294] - BFF on Windows should cancel any remaining timers on finishing campaign
• [BFF-311] - Standalone Windows minimizer is creating zero-length msec files
• [BFF-312] - Minimizer giving up before it should
• [BFF-464] - Issues with OSX installer results directories
• [BFF-485] - tools/repro.py doesn't support OS X
• [BFF-512] - BFF should handle KeyboardInterrupts more gracefully
• [BFF-515] - Permission denied when doing a zip-based minimization to string
• [BFF-521] - Probability out of range in certfuzz.scoring.multiarmed_bandit.arms.base
• [BFF-533] - Pin_calltrace doesn't play nice with minimizer
• [BFF-534] - certfuzz.test.fuzztools.test_rangefinder.Test.test_get_ranges fails sometimes
• [BFF-742] - reboot recovery currently broken in develop branch
• [BFF-743] - git-based make_dist.py zip missing "results" directory
• [BFF-745] - BFF creating bff.log in ~/pintool on Linux
• [BFF-748] - Remove Android code from main development branch
• [BFF-751] - use subprocess.check_call instead of subprocess.call
• [BFF-754] - BFF OSX installer isn't including crashwrangler source zip
• [BFF-756] - Allow batch.sh to take arguments
• [BFF-764] - Calltracefile is not defined in minimizer_base.py
• [BFF-766] - Don't install Python on Yosemite
• [BFF-778] - BFF should check for zzuf at startup
• [BFF-792] - tmp_reaper isn't cleaning up symlinks
• [BFF-793] - gdb output mostly empty on Ubuntu 15.04
• [BFF-794] - drillresults broken
• [BFF-796] - zzuf won't compile on Ubuntu 15.04
• [BFF-798] - drillresults can truncate addresses with 64-bit apps
• [BFF-799] - quickstats.sh is broken
• [BFF-800] - repro.py doesn't work
• [BFF-801] - _cache_app probably shouldn't set use_shell=True
• [BFF-802] - repro.py can't find config
• [BFF-803] - restarting the watchdog only makes sense for UbuFuzz
• [BFF-804] - zzuf seeing signal 9 on every gnash iteration
• [BFF-818] - Windows hook isn't detecting crashes
• [BFF-819] - BFF should verify filesystem when checking for already-existing crashes.
• [BFF-821] - drillresults is broken on Windows
• [BFF-824] - UbuFuzz should have python hcluster
• [BFF-825] - UbuFuzz needs X to be able to be zapped
• [BFF-826] - BFF removes bad zip files before it can run them
• [BFF-827] - zzuf copy mode file I/O is redundant in the new BFF architecture
• [BFF-828] - Minimizer should be zip-aware
• [BFF-829] - BFF isn't touching the watchdog file
• [BFF-831] - quickstats.py is broken
• [BFF-834] - Raised errors in minimizer kill the campaign
• [BFF-835] - drillresults shouldn't complain when the debugger file it's looking for doesn't have what it needs
• [BFF-836] - minimizer_plot.py doesn't work
• [BFF-837] - drillresults can use way too much memory
• [BFF-839] - Drop fuzzer doesn't work
• [BFF-840] - BFF attempts to minimize non-minimizeable fuzzers
• [BFF-841] - Linux BFF increments stats twice
• [BFF-843] - When valgrind is disabled, callgrind annotation still runs
• [BFF-844] - BFF raises an exception when the first seedfile is exhausted
• [BFF-845] - BFF doesn't have the capability of removing seedfiles from the set
• [BFF-846] - Debug mode hangs with winpdb stuff installed
• [BFF-847] - Stopping BFF campaign doesn't stop killproc.sh
• [BFF-849] - Total stack corruption: 'AnalyzerEmptyOutputError' is not defined
• [BFF-850] - UbuFuzz needs x86 compatibility libs
• [BFF-851] - EFA is incorr...

BFF v2.8-beta-3

BFF_2.8b3

BFF v2.8-beta-2

BFF_2.8b2

BFF v2.8-beta-1

BFF_2.8b1