-
Notifications
You must be signed in to change notification settings - Fork 618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VZ: Support Kernel Image #2562
VZ: Support Kernel Image #2562
Conversation
When using this change to boot a VM with a kernel image, $ limactl shell docker sudo systemctl status lima-guestagent
● lima-guestagent.service - lima-guestagent
Loaded: loaded (/etc/systemd/system/lima-guestagent.service; enabled; preset: enabled)
Active: active (running) since Thu 2024-08-29 08:52:54 JST; 10min ago
Main PID: 1670 (lima-guestagent)
Tasks: 7 (limit: 19101)
Memory: 7.1M (peak: 8.3M)
CPU: 2.010s
CGroup: /system.slice/lima-guestagent.service
└─1670 /usr/local/bin/lima-guestagent daemon --vsock-port 2222
Aug 29 09:02:14 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:14+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:24 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:24+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:34 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:34+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:44 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:44+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:54 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:54+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:04 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:04+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:14 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:14+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:24 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:24+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:34 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:34+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:44 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:44+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory" |
83c5161
to
c34fa61
Compare
2f81750
to
600632d
Compare
600632d
to
569a551
Compare
0347c2c
to
df917d2
Compare
@norio-nomura Do you have a sample yaml for running with kernel ?? |
Also could you squash the commits for |
68e4237
to
c4af52e
Compare
You can inject kernel related info into hack/inject-cmdline-to-template.sh templates/ubuntu.yaml console=hvc0 e.g. $ hack/inject-cmdline-to-template.sh templates/ubuntu.yaml console=hvc0
kernel_location=https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-vmlinuz-generic
kernel_digest=sha256:69e66230d6acc503082f098fd072a672a804774007209110b353d25bb7007669
cmdline=root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyAMA0 console=hvc0
initrd_location=https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-initrd-generic
initrd_digest=sha256:92acf9f2976b7a8b3b5e9e3f22211d17a6fdb3f2aed981467f91cd6dbfa16ae5
$ limactl start templates/ubuntu.yaml --tty=false
INFO[0000] Terminal is not available, proceeding without opening an editor
INFO[0000] Starting the instance "ubuntu" with VM driver "vz"
WARN[0000] vmType vz: ignoring images[1]: [Kernel Initrd]
INFO[0000] Attempting to download the image arch=aarch64 digest="sha256:5ecac6447be66a164626744a87a27fd4e6c6606dc683e0a233870af63df4276a" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/ubuntu-24.04-server-cloudimg-arm64.img"
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/346ee1ff9e381b78ba08e2a29445960b5cd31c51f896fc346b82e26e345a5b9a/data"
INFO[0000] Attempting to download the kernel arch=aarch64 digest="sha256:69e66230d6acc503082f098fd072a672a804774007209110b353d25bb7007669" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-vmlinuz-generic"
INFO[0000] decompressing .04-server-cloudimg-arm64-vmlinuz-generic with gzip
INFO[0000] Decompressing data
17.39 MiB / 17.39 MiB [------------------------------------------] 100.00% ? p/s
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/98c212aa1b30c23b56134728fa6adf3096bea13b93cf1744bf88d35d488be2e7/data"
INFO[0000] Attempting to download the initrd arch=aarch64 digest="sha256:92acf9f2976b7a8b3b5e9e3f22211d17a6fdb3f2aed981467f91cd6dbfa16ae5" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-initrd-generic"
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/c972a5d4d42628ec1dd6610a5398674a7f85a3bfa916d614064809722e0f4840/data"
INFO[0000] Converting "/Users/norio/.lima/ubuntu/basedisk" (qcow2) to a raw disk "/Users/norio/.lima/ubuntu/diffdisk"
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 213.27 MiB/s
INFO[0017] Expanding to 100GiB
INFO[0017] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:77c747f09853ee3d229d77e8de0dd3c85622537d82be57433dc1fca4493bab95" location="https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz"
INFO[0017] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/21cc8dfa548ea8a678135bd6984c9feb9f8a01901d10b11bb491f6f4e7537158/data"
WARN[0018] [hostagent] GRPC port forwarding is experimental
INFO[0018] [hostagent] hostagent socket created at /Users/norio/.lima/ubuntu/ha.sock
INFO[0018] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/norio/.lima/ubuntu/serial*.log")
INFO[0018] [hostagent] new connection from to
INFO[0018] SSH Local Port: 59900
INFO[0018] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0018] [hostagent] [VZ] - vm state change: running
INFO[0028] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0028] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0028] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0028] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0028] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0028] [hostagent] Guest agent is running
INFO[0028] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0028] [hostagent] Not forwarding TCP 127.0.0.54:53
INFO[0028] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0028] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0028] [hostagent] Not forwarding TCP [::]:22
INFO[0028] [hostagent] Not forwarding UDP 127.0.0.54:53
INFO[0028] [hostagent] Not forwarding UDP 127.0.0.53:53
INFO[0028] [hostagent] Not forwarding UDP 192.168.5.15:68
INFO[0037] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0037] [hostagent] Waiting for the guest agent to be running
INFO[0037] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0037] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0038] READY. Run `limactl shell ubuntu` to open the shell. |
c4af52e
to
72a1cb6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
Please squash the commits
Signed-off-by: Norio Nomura <[email protected]> vz: change `linuxBootLoader()` to return error, then explicitly ignores the error on caller. Signed-off-by: Norio Nomura <[email protected]> vz: add comment to `bootLoader()` Signed-off-by: Norio Nomura <[email protected]> vz: add `errors.Is(err, os.ErrNotExist)` Signed-off-by: Norio Nomura <[email protected]> vz: use `logrus.WithError(err)` Signed-off-by: Norio Nomura <[email protected]> vz: return `err` if `linuxBootLoader()` returns error other than `os.ErrNotExist` Signed-off-by: Norio Nomura <[email protected]> downloader: detect compression by reading magic Beacause the kernel file provided by Ubuntu is gzipped without `.gz` file extension. Signed-off-by: Norio Nomura <[email protected]> inject-cmdline-to-template.sh: add check location existence Some ubuntu kernel images are provided without initrd image. e.g. https://cloud-images.ubuntu.com/minimal/releases/noble/release-20240823/unpacked/ Signed-off-by: Norio Nomura <[email protected]> inject-cmdline-to-template.sh: use `-e` again Signed-off-by: Norio Nomura <[email protected]>
72a1cb6
to
5223ad9
Compare
Thanks! 🙏🏻 |
⚠️ **CAUTION: this is a major update, indicating a breaking change!**⚠️ This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lima-vm/lima](https://github.com/lima-vm/lima) | major | `v0.23.2` -> `v1.0.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lima-vm/lima (lima-vm/lima)</summary> ### [`v1.0.2`](https://github.com/lima-vm/lima/releases/tag/v1.0.2) [Compare Source](lima-vm/lima@v1.0.1...v1.0.2) #### Changes - DNS: - Fixed the host resolver regression in v1.0.0 [#​2939](lima-vm/lima#2939) ([#​2964](lima-vm/lima#2964)) - `limactl create`: - Fixed races during parallel downloads ([#​2903](lima-vm/lima#2903), thanks to [@​nirs](https://github.com/nirs)) - Optimized qcow2-to-raw conversion for vz mode ([#​2933](lima-vm/lima#2933), thanks to [@​nirs](https://github.com/nirs)) - `limactl start-at-login`: - Fixed the support for Linux hosts (systemd) ([#​2943](lima-vm/lima#2943), thanks to [@​kachick](https://github.com/kachick)) - nerdctl: - Updated to [v2.0.1](https://github.com/containerd/nerdctl/releases/tag/v2.0.1) ([#​2966](lima-vm/lima#2966)) - Templates: - Updated to the latest revisions ([#​2936](lima-vm/lima#2936) [#​2953](lima-vm/lima#2953), thanks to [@​tcooper](https://github.com/tcooper)) - Web site: - Added an example of running Lima on GitHub Actions to run commands on non-Ubuntu ([#​2954](lima-vm/lima#2954)): https://lima-vm.io/docs/examples/gha/ - Project: - Invite Nir Soffer ([@​nirs](https://github.com/nirs)) as a Reviewer ([#​2916](lima-vm/lima#2916), thanks to [@​jandubois](https://github.com/jandubois)) Full changes: https://github.com/lima-vm/lima/milestone/51?closed=1 Thanks to [@​SpiffyEight77](https://github.com/SpiffyEight77) [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​kachick](https://github.com/kachick) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​tamird](https://github.com/tamird) [@​tcooper](https://github.com/tcooper) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/12134682585 The sha256sum of the SHA256SUMS file itself is `02ef78494c498ca4180915ba78d5e2fc471ed401f63dfb2b5864c3711f3c0fb2` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.1`](https://github.com/lima-vm/lima/releases/tag/v1.0.1) [Compare Source](lima-vm/lima@v1.0.0...v1.0.1) Reverted the default port forwarder from gRPC to SSH for the stability reason ([#​2864](lima-vm/lima#2864)). This reversion fixes several regressions related to `docker run -p` in Lima v1.0.0 ([#​2859](lima-vm/lima#2859)). Although the gRPC forwarder is faster and has an advanced feature (UDP support), it turned out to be still immature. Set `LIMA_SSH_PORT_FORWARDER=false` to opt-in to the gRPC forwarder. See <https://lima-vm.io/docs/config/port/>. Full changes: https://github.com/lima-vm/lima/milestone/50?closed=1 Thanks to [@​alexandear](https://github.com/alexandear) [@​jandubois](https://github.com/jandubois) [@​norio-nomura](https://github.com/norio-nomura) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11735352652 The sha256sum of the SHA256SUMS file itself is `f5c12d003e25dc46291803a8acae9e9d325a45eca0c1f9f40bd6852ec8ed9be1` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) ### [`v1.0.0`](https://github.com/lima-vm/lima/releases/tag/v1.0.0) [Compare Source](lima-vm/lima@v0.23.2...v1.0.0) With the support from 110+ contributors in 3+ years, the Lima project has finally reached v1.0. 🎉 This release introduces several breaking changes, such as switching the default machine driver from QEMU to VZ for better filesystem performance. The `limactl` CLI is designed to print hints when the user hits those breaking changes. e.g., `limactl create template://experimental/vz` now fails with a hint that suggests using `limactl create --vm-type=vz template://default` instead. 🔴 = Major breaking changes 🟡 = Minor breaking changes - VZ: - Graduate VZ machine driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use VZ by default for new instances on macOS >= 13.5 ([#​1951](lima-vm/lima#1951)) - Support nested virtualization on M3 ([#​2530](lima-vm/lima#2530), thanks to [@​abiosoft](https://github.com/abiosoft)) - Optimize qcow2-to-raw image conversion (lima-vm/go-qcow2reader@v0.1.2...v0.4.0 , thanks to [@​nirs](https://github.com/nirs)) - Support specifying a custom kernel ([#​2562](lima-vm/lima#2562), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - QEMU: - Graduate 9p mount driver from experimental ([#​2758](lima-vm/lima#2758)) - 🔴 Use 9p by default for most templates ([#​1953](lima-vm/lima#1953), [#​2822](lima-vm/lima#2822)) - riscv64: switch from u-boot to EDK2 ([#​2592](lima-vm/lima#2592)) - Network: - Graduate user-v2 network driver from experimental ([#​2758](lima-vm/lima#2758)) - Support UDP port forwarding ([#​2411](lima-vm/lima#2411), thanks to [@​balajiv113](https://github.com/balajiv113)) - 🔴 Strictly require `socket_vmnet` binary to be owned by root ([#​2734](lima-vm/lima#2734)) - SSH: - 🟡 Disable `ssh.loadDotSSHPubKeys` by default ([#​2706](lima-vm/lima#2706)) - YAML: - Support generating jsonschema ([#​2306](lima-vm/lima#2306), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - Support specifying `param` for provisioning scripts ([#​2570](lima-vm/lima#2570), thanks to [@​jandubois](https://github.com/jandubois)) - Support specifying `minimumLimaVersion` and `vmOpts.qemu.minimumVersion` ([#​2659](lima-vm/lima#2659), thanks to [@​jandubois](https://github.com/jandubois)) - Support template expansion in mounts ([#​2588](lima-vm/lima#2588), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - `limactl` CLI: - Add `limactl tunnel` command so as to allow the host to join the guest network ([#​2710](lima-vm/lima#2710)) - Add `--log-format=json` ([#​2584](lima-vm/lima#2584), thanks to [@​nirs](https://github.com/nirs)) - `limactl prune`: Add `--keep-referred` ([#​2569](lima-vm/lima#2569), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - nerdctl: - Updated to [v2.0.0](https://github.com/containerd/nerdctl/releases/tag/v2.0.0) ([#​2178](lima-vm/lima#2178)) - rootless: allocate 1G subuids from 524288 (0x80000) for new users ([#​2725](lima-vm/lima#2725)) - Templates: - 🔴 `experimental/vz`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🟡 `experimental/{riscv64, armv7l}`: Merged into the `default` template ([#​2730](lima-vm/lima#2730), [#​2736](lima-vm/lima#2736)) - 🔴 `vmnet`: Removed in favor of `limactl create --network=lima:shared template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/net-user-v2`: Removed in favor of `limactl create --network=lima:user-v2 template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `experimental/9p`: Removed in favor of `limactl create --mount-type=9p template://default` ([#​2736](lima-vm/lima#2736)) - 🟡 `experimental/virtiofs-linux`: Removed in favor of `limactl create --mount-type=virtiofs template://default` ([#​2736](lima-vm/lima#2736)) - 🔴 `alpine`: Renamed to `alpine-iso` ([#​2704](lima-vm/lima#2704)) - 🔴 `alpine-image`: Renamed to `alpine` ([#​2704](lima-vm/lima#2704)) - `archlinux`: Demoted from Tier 1 to Tier 2 ([#​2717](lima-vm/lima#2717), [#​2823](lima-vm/lima#2823)) - `default`, `ubuntu`, ...: Updated to Ubuntu 24.10. The older versions are available as `ubuntu-20.04`, `ubuntu-22.04`, and `ubuntu-24.04` ([#​2755](lima-vm/lima#2755), [#​2795](lima-vm/lima#2795)) - `fedora`: Updated to Fedora 41 ([#​2821](lima-vm/lima#2821), [#​2822](lima-vm/lima#2822), thanks to [@​subpop](https://github.com/subpop)) - `opensuse`: Renamed to `opensuse-leap`. Still aliased as `opensuse` ([#​2612](lima-vm/lima#2612), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `experimental/opensuse-tumbleweed`: Support aarch64 ([#​2613](lima-vm/lima#2613), thanks to [@​afbjorklund](https://github.com/afbjorklund)) - `hack/update-template.sh` is added for automating updates ([#​1347](lima-vm/lima#1347), thanks to [@​norio-nomura](https://github.com/norio-nomura)) - Project: - Invite Norio Nomura ([@​norio-nomura](https://github.com/norio-nomura)) as a Reviewer ([#​2567](lima-vm/lima#2567)) Full changes: https://github.com/lima-vm/lima/milestone/47?closed=1 Thanks to [@​AdamKorcz](https://github.com/AdamKorcz) [@​Mr-Sunglasses](https://github.com/Mr-Sunglasses) [@​SmartManoj](https://github.com/SmartManoj) [@​YorikSar](https://github.com/YorikSar) [@​abiosoft](https://github.com/abiosoft) [@​afbjorklund](https://github.com/afbjorklund) [@​alexandear](https://github.com/alexandear) [@​balajiv113](https://github.com/balajiv113) [@​hasan4791](https://github.com/hasan4791) [@​jandubois](https://github.com/jandubois) [@​nirs](https://github.com/nirs) [@​norio-nomura](https://github.com/norio-nomura) [@​pvdvreede](https://github.com/pvdvreede) [@​subpop](https://github.com/subpop) [@​tsukasaI](https://github.com/tsukasaI) #### Usage ```console [macOS]$ limactl create [macOS]$ limactl start ... INFO[0029] READY. Run `lima` to open the shell. [macOS]$ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/11695321667 The sha256sum of the SHA256SUMS file itself is `4bd200a163111fe78c6f3e6de405113d416053802fe1507597f9a42f89a98c90` . *** Release manager: [@​AkihiroSuda](https://github.com/AkihiroSuda) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Similar to QEMU's kernel image support.
When using
VZLinuxBootLoader
(https://developer.apple.com/documentation/virtualization/vzlinuxbootloader), theVirtualization.framework
does not provide EFI to the VM. As a result, the Linux kernel does not recognize a device available asrtc0
, which means the correct time cannot be obtained during the kernel boot. Consequently, the VM's time is not corrected untiltimesync.service
starts viacloud-init
, which can cause issues likeapt-get
failures. If, for any reason,timesync.service
fails to correct the time,apt-get
in the provisioning script might fail, causinglimactl
to perceive the VM boot as failed.I'm hesitant to add tests for this feature in CI because I'm unsure of the reasons why
timesync.service
might fail to correct the time.I believe
inject-cmdline-to-template.sh
will be helpful when testing this feature.