Stability of Onyx booted off hard drive?

[petershh]

I think it would be nice to have a copy of Onyx installed on a (virtual) hard drive. It would be useful for e.g. porting userspace stuff.

But IIRC there are some stability issues in such setups. What are those issues?

[heatd]

1. Writeback has historically been iffy (I think it's mostly okay now)[2]
2. Some filesystem operations have resulted in races or crashes when stressed[1]
3. Doing operations on the raw block device (say, doing mkfs) and then mounting them isn't interacting properly[2]

[1] I think it is okay-ish now, but i'm still stressing things using stress-ng and other stressing tools and manually verifying consistency using fsck on the host OS. KCSAN is really helpful here, I'm trying to see if I can upstream it ASAP and then run it on the whole kernel
[2] Sooo, block bufs are a fun thing. They kinda mirror linux buffer_heads and UNIX struct bufs. That model is kinda crappy. Basically, we set up a block_buf for each block in a page. This ends up meaning that writing to a raw block dev will set up block_bufs for 512 byte blocks (sector-sized), then mounting a filesystem will read the superblock and change the block device. This will fuck up the block device's page cache. We need to truncate the bdev's page cache (clearing all old-sized block_bufs) when switching block sizes. This also ties in with [1]. I kind of want to eventually get rid of block_bufs, there's also a fun race where you can lose data.

[heatd]

Note that booting off a hard drive is fine, I do that regularly (there are several targets to that effect in the Makefile, that use the just built .iso's kernel and initrd to bootstrap into the hard drive). Installing is iffier

[heatd]

Ok, updates on the installing bit:
Merged:
#112 #111 were helpful in finding IO bugs and adding missing features

To merge:
#113 speeds up hard drive accesses
#114 lets us partition the drive, mkfs and copy a bunch of files (with no problems according to fsck)

Relevant ports:
GNU Parted
heatd/onyx-package-tree@aa6d03c

e2fsprogs
heatd/onyx-package-tree@6b996f3

Basically, with these last two PRs merged, installing a system should be possible, given a GRUB port (to be done). A package manager would be nice, but not needed, we can achieve the same-ish effect by decompressing and tar xf'ing tarballs to the new root (like good old UNIX did). Exciting times ahead!

[heatd]

Side-notes: deciding on a package manager would be cool. Building ports is tedious atm with my hacked-up python script and crappy shell scripts. RPM? DPKG? pacman? See which of these are good at cross-compiling. And in the future make the base system a bunch of packages too.

Also, page reclamation is needed. If lots of IO is done, we risk running out of pages and dying. I do not trust the system's OOM handling, at all.

Also, umount is still not implemented. This sucks, atm the best way of knowing that the filesystems are clean is just to sync(1) once or twice

[heatd]

Update 2: #113 and #114 are getting merged tonight. I got a GRUB port with a working grub-install. Still need to finish it up and generate the patches (and upstream it to onyx-package-tree). Yay!