Split lkl_start_init into 2 stages

[jxyang]

We should separate lkl_start_init into 2 stages:

• stage 1: set up the kernel to a point where syscalls from libc can be serviced. Pre-stage1, no calls to libs that triggers a syscall is allowed. Other functions, such as snprintf, are still possible.
• stage 2: set up user facing functionalities of lkl, such as lkl_mount_disks. In this stage, we are free to call all libc functions.

So the general initialization flow becomes:

1. lthread setup
2. lkl init stage 1
3. libc init
4. lkl init stage 2
5. other initializations depending on libc

[jxyang]

@davidchisnall How do I link this to the relayering project?

[davidchisnall]

I think this is mostly captured already in #183 and #184.

[davidchisnall]

I'd like to replace all libc calls prior to kernel entry. For example, snprintf can easily be replaced with oe_snprintf (see #151). Once the relayering is done, I intend to introduce a new build system that will check that the SGX-LKL kernel component, when linked, does not have any undefined symbols. The userspace bit will then be linked separately and checked to have only the lkl_syscall symbol undefined, to prevent layering regressions.

Longer term, musl and glibc ports will live in a separate filesystem bundled with SGX-LKL and be loaded depending on the header in the executable. At that point, having libc sybmols referenced from the kernel component definitely won't work.

[jxyang]

Short term, and more realistically, I think we could have both musl and lkl co-exists in the kernel, but eliminate all code related to lthread, enclave, and lkl from musl (except for the syscall interface). We still have a nice separation, but probably not as pure as we hoped.

[davidchisnall]

Closing this as a duplicate of #183 and #184.