Google Summer of Code 2014

This page contains project ideas for Google Summer of Code 2014! If you're interested in making a proposal based on these ideas, please send a list to the mailing list or the mentor listed for the idea.

Ideas

Memory allocator enhancement

Unlike Linux, OSv does not have any kind of structure describing a page. That means some way of figuring out characteristics about the page / allocation is needed.

Currently, free(v) uses the offset of an objectwithin a page (o = v & 0xfff) to determine which allocator is responsible for the page:

• if o == 0, it is a large object (> page size)
• if o == 8, it is a an early allocator object
• otherwise, it is a mempool object, with the pool page header residing at the beginning of the page (v - o)

this scheme has a number of issues:

1. allocating objects that are exactly page sized waste two pages
2. it is hard to align objects that are between half a page and a page
3. the header for a large object is sized an entire page (the expected waste is half a page on average)

We can improve this by using information other than the offset within the page to determine the allocator. If we map physical memory multiple times (all aliases of each other), we can use virtual address bits 44:46 to determine the allocator.

The value of v[44:46] can be used as follows:

0 - mempool allocator 1 - page allocator (for objects between half a page and a page) 2 - half-page allocator (for objects exactly half a page, useful for network buffers) 3 - large objects 4-5 reserved 6 - early allocator objects 7 - debug allocator (using a non identity virtual:physical mapping)

The virtual memory map will now be

  0000_0000_0000_0000 - 0000_7fff_ffff_ffff kernel code and application virtual memory
  ffff_8000_0000_0000 - ffff_8fff_ffff_ffff mempool (sub-page) objects
  ffff_9000_0000_0000 - ffff_9fff_ffff_ffff page objects
  ffff_a000_0000_0000 - ffff_afff_ffff_ffff halfpage objects
  ffff_b000_0000_0000 - ffff_bfff_ffff_ffff large objects (> 1 page)
  ffff_e000_0000_0000 - ffff_efff_ffff_ffff early allocator objects
  ffff_f000_0000_0000 - ffff_ffff_ffff_ffff debug allocator objects

More details an follow up discussions at in https://github.com/cloudius-systems/osv/issues/87

Difficulty: Medium

Required skills: Low-level memory management, C++

Mentor: Avi Kivity [email protected]

Improve runtime support

There are a number of things we need to do to improve our ability to run runtimes other than the JVM. Part of it consists of enabling other runtimes to be executed inside OSv, part of it consists of improving our ability to run without a runtime at all. Currently, only the JVM and mruby are fully supported.

The student working on this will have to bootstrap the proper runtime and hook it into our module system, so that it becomes as easy to run other runtimes as it is to run java now.

• Go language runtime support

Although we are interested in expanding our runtime list to everything that is relevant, go is one of our top priorities. Our new management module, capstan (https://github.com/cloudius-systems/capstan) is written in go, and we expect that to gain relevance.

OSv is implementing the POSIX API layer on a need-to-do basis. The first task of porting a new runtime is identifying which calls are needed for go that are not implemented by OSv, and then of course, implementing them.

• Running unmodified binaries

Due to initial technical constraints, we do not have the ability to run an ELF binary. We have our own ELF loader and from there, we execute a shared object (.so). While this works, this means native compiled applications that wants to be ran on OSv need to be recompiled as a shared library.

It is technically possible to lift that limitation, by:

• Loading a full elf and reading its list of mappings
• If the kernel mapping overlaps the binary (it probably does), remap the kernel somewhere else, where the application left a hole unfilled
• Audit all the places in which the kernel symbols are used directly: those places will access the old addresses, and will need to be fixed.

Difficulty: Easy to Medium Required skills: POSIX, C++

Virtual light weight RDMA

Difficulty: Hard

Required skills: virtio

Shared memory device (Nahanni code)

Xen PV or Xen PVH mode

Currently, our Xen support is restricted to HVM guests, which means hardware virtualization extensions are needed. However, that is not Xen's main mode of operation. That causes us to lag behind in Xen-centric deployments like Amazon.

There is a very basic pv code in OSv, that go as far as setting up the start-of-day layout and page tables (although the later is not merged)

The student tackling this project will need to implement all the relevant hypercalls that will enable OSv to boon on top of Xen, and also make sure his implementation is efficient

Difficulty: Medium

Required skills: x86 architecture, Xen.

Mentor: Glauber Costa [email protected]

Cloud foundry support for OSv (instead of only containers today)

Symbolic Links support

ZFS, the default disk-based file system in OSv, supports symbolic links, but the respective functions (zfs_symlink and zfs_readlink) are not integrated into the vnode operations list (VNOPS) from our Virtual File System (VFS) layer. To do that, you will have to integrate both functions into the ZFS VNOPS (both located at '.../zfs/zfs_vnops.c'), and add respective entries into the VNOPS structure which is located at '/include/osv/vnode.h'. The final step is to implement the function symlink, and the system calls sys_symlink and sys_readlink (partially implemented), readlink is already implemented. Another important detail is that these functions must be POSIX compliant. I would recommend using manpages as they are very descriptive about the functionality and error codes.

Difficulty: Medium

Required skills: File systems, C/C++.