Use architecture-native images and binaries (arm64 for mac M1)

[rfay]

I was caught by surprise to see qemu running things on my arm64 M1 mac.

qemu is a wonderful concoction, but it's not something you want to rely on for real work. It has tremendous problems with reliabilities/crashes and also with performance.

But there's not really a reason I understand not to use native binaries and images. The project is written in golang, it's easy to compile to multiple target architectures, and it's easy to build native docker images.

Is there already work going on in this area?

[jandubois]

But there's not really a reason I understand not to use native binaries and images.

I don't know what you mean by "native binaries". The goal is to run a container runtime, like containerd, and so far they only run on Linux, so on macOS you need a virtual machine to provide the Linux environment. It doesn't have to be qemu, but you need some kind of hypervisor, and given that Lima is an open source project, the hypervisor should also be freely available under an open source license as well (which rules out Parallels, VMware Fusion, etc).

Given that qemu uses the Apple hypervisor framework to run native binaries at native speed, I don't understand the performance issue.

But really, what is the alternative to run containerd on macOS?

[afbjorklund]

@jandubois : on ubuntu, they are installed with the binfmt-support package (a dependency of qemu-user-static)

[afbjorklund]

@rfay : yet all your comments were about qemu-user ? Anyway, lima should support both of them - i.e. if you need it

[rfay]

No, really the comments were a worry that this was using qemu-system to do cross-architecture runs, but from all the kind responses here I guess that's not the case. I look forward to getting more familiar with this system.

[afbjorklund]

The virtual machine will have to run with qemu-system, but hopefully it will be accelerated (hvf/kvm)...
The containers don't have to run qemu-user, if they have right architecture.

[rfay]

My original worry was that the VM was set up as amd64 and would have to run cross-architecture on an arm64 machine.

[afbjorklund]

But really, what is the alternative to run containerd on macOS?

The alternative would be jails (from FreeBSD), but they are not there.

Then again, WSL 1 ran containers (on Windows). Look how far it got ?

So I guess your best bet is to run a real native binary, i.e. darwin