most minimal host

[vilvo]

    Is this really the minimal we want? This is minimal for a generic headless device, I envisaged us creating minimal-minimal.nix, or barely-enough-to-boot.nix profiles, which we could contribute back.

Originally posted by @brianmcgillion in #41 (review)

[jenninikko]

Documented here: https://tiiuae.github.io/ghaf/architecture/adr/minimal-host.html

[remimimimimi]

As it would be inconvenient to do all those experiments in the ghaf repository, I have yet to create a draft pull request. All of my experiments (which were committed) are located here and here.

Experiments

Ghaf minimal

The first repository consists of several branches, each of which represents some experiment:

gist

Implements this gist in nix.

gist-glibc

Refines branch above with the use of glibc instead of musl.

gist-systemd

Uses systemd for initialization instead of a hand-written script.

nixos-toplevel

Keeps the essential packages for NixOS. Removes almost everything, and the remaining system builds but not working.

It shows how everything indirectly depends upon top-level derivation. Dependencies could be tracked buy only through source code. But when I looked into it, I saw that many stuff are hard-coded.

For example, there's no easy way to replace coreutils and util-linux with busybox because nixpkgs depend on it heavily. Also, Perl used all over NixOS for different scripts: install grub, activation scripts, and switch to configuration.

Ghaf not

Here I tried to fork not-os to essentially reimplement low-level components of NixOS and then plug high-level modules in.

Next steps

We have the following requirements: easy-maintainable, small TCB in the resulting image. The problem is that it is currently impossible for both of them. The maximum that we can have right now without forking NixOS or reimplementing low-level components is to modify the systemd package.

If we choose to maintain our code base, then the most sensible steps will be the following:

• Analyze modules to separate them into low-level and high-level.
• Reimplement low-level modules and plug high-level ones on top of them.

We'll have the smallest possible TCB, but we'll need to support our code base, which may break over time.

The other way is to keep things simple and stay with the current NixOS code base with minor modifications for the image build scripts to eliminate Nix on the host.

The last approach seems more sensible in the long term because we can patch things gradually, as CVE's will be found for some of the packages we'll use.

There are currently two ways to patch packages. Either with system.replaceRuntimeDependencies, which will modify the Nix store and doesn't require rebuilding of the package (work impurely), or with overlays, which require a full system rebuild.