Running OSv on Firecracker

The easiest way to run OSv on Firecracker is to use a Python script firecracker.py. The firecracker.py automates the process of launching firecracker VMM executable in order to start OSv micro-VM. It does it by either submitting necessary REST api calls over UNIX domain socket to a running firecracker process or creating a configuration file and passing its path as an argument to execute firecracker (api-less mode). For insight of why this script does what it does look at Firecracker quick start guide.

Step 0: Prerequisites

In order to run OSv on Firecracker you need bare-metal machine (AWS i3.metal or other Nitro-based EC2 instance should work as well) with Linux installed and KVM enabled and OSv build environment to build arbitrary images. Alternatively one can use Mikelangelo capstan to build images. In short, if one has already OSv development environment ready, firecracker should work out of the box. The firecracker.py will automatically download and install the latest version of the firecracker unless you point to an alternative location of firecracker executable through FIRECRACKER_PATH variable.

Step 1: Building and running

There is nothing special about building OSv images to run them on Firecracker. However please bear in mind that Firecracker boots OSv by directly executing build/release/loader.elf (which conforms to Linux vmlinux 64-bit ELF format) instead of usr.img used by QEMU and other hypervisors. The build/release/usr.raw file (the raw version of usr.img) can be used to mount a zfs or rofs disk with an application.

./scripts/build image=native-example
./scripts/firecracker.py 

Please note that firecracker.py provides a subset of functionality similar to run.py with almost identical options:

./scripts/firecracker.py --help
usage: firecracker [-h] [-c VCPUS] [-m MEMSIZE] [-e CMD] [-i CMD] [-k CMD]
                   [-n] [-b BRIDGE] [-V] [-l] [-p PHYSICAL_NIC]

optional arguments:
  -h, --help            show this help message and exit
  -c VCPUS, --vcpus VCPUS
                        specify number of vcpus
  -m MEMSIZE, --memsize MEMSIZE
                        specify memory: ex. 1G, 2G, ...
  -e CMD, --execute CMD
                        overwrite command line
  -i CMD, --image CMD   path to disk image file. defaults to
                        ../build/release/usr.img
  -k CMD, --kernel CMD  path to kernel loader file. defaults to
                        ../build/release/loader-stripped.elf
  -n, --networking      needs root to setup tap networking first time
  -b BRIDGE, --bridge BRIDGE
                        bridge name for tap networking
  -V, --verbose         pass --verbose to OSv, to display more debugging
                        information on the console
  -l, --api_less        do NOT use socket-based API to configure and start OSv
                        on firecracker
  -p PHYSICAL_NIC, --physical_nic PHYSICAL_NIC
                        name of the physical NIC (wired or wireless) to
                        forward to if in natted mode

Networking

The difference is that unlike run.py, firecracker.py does not enable networking automatically. Instead you have to explicitly pass -n option to make it automatically setup TAP device and pass IP address to OSv:

./scripts/firecracker.py -n -c 2
The tap interface fc_tap0 not found!. Needs to set it up
[sudo] password for XXXXX:
net.ipv4.conf.fc_tap0.proxy_arp = 1
net.ipv6.conf.fc_tap0.disable_ipv6 = 1
OSv v0.52.0-51-g08027169
eth0: 172.16.0.2
2019-03-05T22:58:56.070439724 [anonymous-instance:WARN:vmm/src/lib.rs:1080] Guest-boot-time =  15607 us 15 ms,  19798 CPU us 19 CPU ms
Starting http server to listen on port 3000 ...