Skip to content

Release 0.54.0 "Dove"
Compare
Choose a tag to compare
@wkozaczuk wkozaczuk released this 16 Sep 11:51
· 937 commits to master since this release
v0.54.0
3e658b8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

This new release of OSv focuses on improving Linux compatibility and tooling aimed to make it possible to run unmodified Linux apps on OSv "as-is".

Overview

From the beginning, OSv was designed to implement a subset of Linux POSIX API superset. But until this release most Linux applications had to be re-compiled from source as shared libraries or some, like Java, rely on OSv version of /usr/bin/java wrapper to run. This meant that one could NOT run a Linux executable "as is". In other words, OSv has always been Linux-compatible at source level but not at binary level.

This release offers a breakthrough and allows running unmodified Linux position-independent executables (so-called "pies") and position-dependant executables "as-is" as long as they do not use "fork/execve" or other unsupported Linux API. It means that very often one can take a binary from Linux host and run it on OSv without having to locate the source code on the Internet and build it as a shared library.

In addition, this release makes OSv more Linux-compatible from another end - booting on a hypervisor. The previous release 0.53 made OSv kernel "look like" ELF64 uncompressed Linux kernel. The new release 0.54 has enhanced OSv loader to "look like" vmlinuz and thus allow booting on Docker's Hyperkit on OSX. The OSv loader has also been enhanced to boot as Linux ELF64 PVH/HVM loader on QEMU with --kernel option.

Highlights

Linux compatibility

  • Applications
    • Enhanced getopt family of functions to work correctly with both position-independent executables and position-dependent executables in order to allow receiving program arguments
    • Enhanced dynamic linker to be capable of executing position-dependent executables
    • Mapped kernel higher in virtual memory - from 0x00200000 to 0x40200000 (2nd GiB) in order to make space for position-dependent executables
    • Added new GNU libc extensions: error(), __prognames and __progname_full
    • Added missing pseudo-files to procfs and minimal implementation of sysfs in order to support libnuma to allow programs like ffmpeg using x265 codec run on OSv "as-is"
    • Encanced /proc/self/maps to include i-node number and device ID to support GraalVM apps with isolates
    • Enhanced epoll_pwait() implementation
    • Improved dynamic linker by making it:
      • Ignore old version symbols so that new version symbols are resolved correctly instead
      • Delay resolving symbols found missing during relocate_rela() phase for certain relocation types to allow more unmodified Linux executables run on OSv
      • Handle DT_RUNPATH
  • Booting
    • Added vmlinuz-compatible version of the kernel to allow OSv boot on Docker's Hyperkit
    • Enhanced loader to support PVH/HVM boot to allow OSV run on QEMU with --kernel option
    • Added support of QEMU 4.x
    • Enhanced HPET driver to support 32-bit main counter

Filesystem improvements

  • VFS
    • Hardened implementation of open()/sys_open()/task_conv() to handle null path
    • Enhanced __fxstata to handle AT_SYMLINK_NOFOLLOW
  • RAMFS
    • Greatly improved speed of write/append operations
    • Fixed bugs
      • Delay freeing data until i-node closed
      • Keep i-node number the same

Tools

  • Added script manifest_from_host.sh to allow building images from artifacts on Linux host “as-is” without need to compile
  • Added script build-capstan-mpm-packages to create capstan MPM packages
  • Added Ubuntu- and Fedora-based Docker files to help create build and test environment
  • Enhanced test.py to allow executing unit tests on Firecracker

Bugs and other enhancements

  • Fixed sem_trywait() that for example allows Java 12 run properly on OSv
  • Improved memory utilization by using memory below the kernel
  • Introduced new command line suffix ! allowing to force termination of lingering threads
  • Revamped building of the cli and httpserver apps to use OpenSSL 1.1 and Lua 5.3 and minimize compilation
  • Tweaked OSv code to support compilation by GCC 9

Improved Documentation

  • Refreshed main README
  • OSv-apps
  • Scripts  

Apps

  • Added number of *-from-host apps that demonstrate building images out of binaries from Linux host:
    • Java
    • Python
    • Node
    • Lua
    • Ffmpeg 
  • Added demo app - openjdk12-jre-from-docker that creates an image out of a Docker image
  • Added demo app that demonstrates running GraalVM isolates
  • Added an example of a basic mono app
  • Improved support of Golang PIEs

Closed issues

  • #1050 - Can't run anything with 1.01G of memory
  • #1049 - tst_huge hangs with memory over 1GB.
  • #1048 - VM with memory larger than 4GB doesn't boot
  • #1043 - Map kernel higher in virtual memory
  • #1039 - Handle new DT_RUNPATH in object::load_needed()
  • #1035 - iperf3 fails with exception nested to deeply on ROFS/RamFS image
  • #1034 - Build failures when build directory's pathname has a space
  • #1031 - The graalvm-example fails with graalvm 1.0.0-rc13
  • #1026 - golang-pie-httpserver crashes on control-C
  • #1023 - Ignore missing symbols when loading objects with BIND_NOW in relocate_rela()
  • #1022 - lua package requires openssl 1.0
  • #1012 - Improve physical memory utilization by using memory below 2MB
  • #884 - slow write/append to files on ramfs
  • #689 - PIE applications using "optarg" do not work on newer gcc
  • #561 - OSv failed to run a pthread application.
  • #534 - imgedit.py can't always connect to qemu-nbd
  • #305 - Fail to run iperf3 on osv
  • #190 - Allow running a single unmodified regular (non-PIE) Linux executable
  • #34 - Mono support

Commits by author

KANATSU Minoru (1):
Nadav Har'El (9):
Waldemar Kozaczuk (86):

Acknowledgments

We want to thank all contributors to the project. But the special thanks go to:

  • Nadav Har’El for contributing and reviewing many patches and providing guidance for many others
  • Waldemar Kozaczuk for contributing most patches
Assets 20
Loading