KVM entry failed error in the linux boot after TPM_CC_SelfTest

[cclaudio]

Description

The guest fails late in the kernel guest boot when a TPM_CC_SelfTest is issued by the kernel. Since we are booting with multiple vcpus, I suppose the SVSM VTPM_REQUEST handler is not multi-thread safe.

This does not happen on every boot, but happened on 2/5 boots.

The guest boots fine with only 1 vcpu.

How to reproduce it:

Build the svsm.bin from either the dbuono-allocator-fix or svsm-vtpm-preview branches:

make FEATURES=verbose

Follow the README.md to build the other dependencies (make sure VTPM=1 is provided) and run the guest:

sudo ./launch-qemu.sh -smp 4 -hda ccguest.qcow2 -svsm ../svsm.bin -sev-snp -svsmcrb -allow-debug -ssh-forward

Partial message log of the guest kernel

[    1.270287] EISA bus registered
[    1.271409] cpuidle: using governor ladder
[    1.271938] cpuidle: using governor menu
[    1.273063] SEV: SNP running at VMPL1.
[    1.273063] SEV: Using SNP CPUID table, 31 entries present.
[    1.274289] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    1.275184] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xb0000000-0xbfffffff] (base 0xb0000000)
[    1.276311] PCI: MMCONFIG at [mem 0xb0000000-0xbfffffff] reserved in E820
[    1.277139] PCI: Using configuration type 1 for base access
[    1.279444] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    1.280546] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    1.280546] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    1.282353] fbcon: Taking over console
[    1.282828] ACPI: Added _OSI(Module Device)
[    1.283349] ACPI: Added _OSI(Processor Device)
[    1.283894] ACPI: Added _OSI(3.0 _SCP Extensions)
[    1.284461] ACPI: Added _OSI(Processor Aggregator Device)
[    1.285109] ACPI: Added _OSI(Linux-Dell-Video)
[    1.286292] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    1.286934] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    1.289088] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    1.290784] ACPI: Interpreter enabled
[    1.291251] ACPI: PM: (supports S0 S3 S4 S5)
[    1.291769] ACPI: Using IOAPIC for interrupt routing
[    1.292420] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    1.293622] ACPI: Enabled 2 GPEs in block 00 to 3F
[    1.296215] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    1.296981] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    1.298117] acpi PNP0A08:00: _OSC: platform does not support [LTR]
[    1.298364] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability]
[    1.299590] PCI host bridge to bus 0000:00
[    1.300093] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    1.300901] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    1.302285] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    1.303183] pci_bus 0000:00: root bus resource [mem 0x80000000-0xafffffff window]
[    1.304105] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[    1.305008] pci_bus 0000:00: root bus resource [mem 0x800000000-0xfffffffff window]
[    1.305920] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.306336] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000
[    1.307662] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000
[    1.309715] pci 0000:00:01.0: reg 0x10: [mem 0xc0000000-0xc0ffffff pref]
[    1.312034] pci 0000:00:01.0: reg 0x18: [mem 0xc1032000-0xc1032fff]
[    1.316813] pci 0000:00:01.0: reg 0x30: [mem 0xffff0000-0xffffffff pref]
[    1.317694] pci 0000:00:01.0: BAR 0: assigned to efifb
[    1.318315] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    1.320412] pci 0000:00:02.0: [1af4:1048] type 00 class 0x010000
[    1.322283] pci 0000:00:02.0: reg 0x14: [mem 0xc1031000-0xc1031fff]
[    1.324478] pci 0000:00:02.0: reg 0x20: [mem 0x800000000-0x800003fff 64bit pref]
[    1.326597] pci 0000:00:03.0: [8086:100e] type 00 class 0x020000
[    1.328225] pci 0000:00:03.0: reg 0x10: [mem 0xc1000000-0xc101ffff]
[    1.329802] pci 0000:00:03.0: reg 0x14: [io  0x6040-0x607f]
[    1.337206] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100
[    1.338675] pci 0000:00:1f.0: quirk: [io  0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO
[    1.339929] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601
[    1.343863] pci 0000:00:1f.2: reg 0x20: [io  0x6080-0x609f]
[    1.344970] pci 0000:00:1f.2: reg 0x24: [mem 0xc1030000-0xc1030fff]
[    1.346641] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500
[    1.348898] pci 0000:00:1f.3: reg 0x20: [io  0x6000-0x603f]
[    1.350811] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[    1.351624] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[    1.353418] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[    1.354224] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[    1.354364] ACPI: PCI: Interrupt link LNKE configured for IRQ 10
[    1.355164] ACPI: PCI: Interrupt link LNKF configured for IRQ 10
[    1.355993] ACPI: PCI: Interrupt link LNKG configured for IRQ 11
[    1.356816] ACPI: PCI: Interrupt link LNKH configured for IRQ 11
[    1.357534] ACPI: PCI: Interrupt link GSIA configured for IRQ 16
[    1.358290] ACPI: PCI: Interrupt link GSIB configured for IRQ 17
[    1.359019] ACPI: PCI: Interrupt link GSIC configured for IRQ 18
[    1.359805] ACPI: PCI: Interrupt link GSID configured for IRQ 19
[    1.360532] ACPI: PCI: Interrupt link GSIE configured for IRQ 20
[    1.361224] ACPI: PCI: Interrupt link GSIF configured for IRQ 21
[    1.362289] ACPI: PCI: Interrupt link GSIG configured for IRQ 22
[    1.362973] ACPI: PCI: Interrupt link GSIH configured for IRQ 23
[    1.364131] iommu: Default domain type: Translated 
[    1.364131] iommu: DMA domain TLB invalidation policy: lazy mode 
[    1.364131] pci 0000:00:01.0: vgaarb: setting as boot VGA device
[    1.364131] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    1.366284] pci 0000:00:01.0: vgaarb: bridge control possible
[    1.366971] vgaarb: loaded
[    1.367478] SCSI subsystem initialized
[    1.367950] ACPI: bus type USB registered
[    1.367950] usbcore: registered new interface driver usbfs
[    1.370287] usbcore: registered new interface driver hub
[    1.370900] usbcore: registered new device driver usb
[    1.371501] pps_core: LinuxPPS API ver. 1 registered
[    1.372103] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    1.373169] PTP clock support registered
[    1.373642] EDAC MC: Ver: 3.0.0
[    1.374422] Registered efivars operations
[    1.374848] NetLabel: Initializing
[    1.374874] NetLabel:  domain hash size = 128
[    1.375410] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    1.376098] NetLabel:  unlabeled traffic allowed by default
[    1.376753] PCI: Using ACPI for IRQ routing
[    1.458655] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    1.458945] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[    1.463369] clocksource: Switched to clocksource kvm-clock
[    1.471054] VFS: Disk quotas dquot_6.6.0
[    1.471541] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.472403] AppArmor: AppArmor Filesystem Enabled
[    1.472947] pnp: PnP ACPI init
[    1.473428] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved
[    1.474475] pnp: PnP ACPI: found 6 devices
[    1.484168] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    1.485205] NET: Registered PF_INET protocol family
[    1.485840] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    1.487134] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    1.488154] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    1.489092] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    1.489967] TCP: Hash tables configured (established 32768 bind 32768)
[    1.490770] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    1.491529] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    1.492367] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    1.493003] NET: Registered PF_XDP protocol family
[    1.493542] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    1.494242] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    1.494945] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    1.495706] pci_bus 0000:00: resource 7 [mem 0x80000000-0xafffffff window]
[    1.496464] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]
[    1.497223] pci_bus 0000:00: resource 9 [mem 0x800000000-0xfffffffff window]
[    1.498092] PCI: CLS 0 bytes, default 64
[    1.498559] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.498653] Trying to unpack rootfs image as initramfs...
[    1.499281] software IO TLB: mapped [mem 0x000000006c9de000-0x000000007bf5e000] (245MB)
[    1.506989] SEV: SNP guest platform device initialized.
[    1.507863] Initialise system trusted keyrings
[    1.508378] Key type blacklist registered
[    1.509063] workingset: timestamp_bits=36 max_order=20 bucket_order=0
[    1.510606] zbud: loaded
[    1.511120] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    1.512118] fuse: init (API version 7.36)
[    1.512911] integrity: Platform Keyring initialized
[    1.516365] Key type asymmetric registered
[    1.516836] Asymmetric key parser 'x509' registered
[    1.866644] Freeing initrd memory: 91640K
[    1.873589] alg: self-tests for CTR-KDF (hmac(sha256)) passed
[    1.875024] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    1.876644] io scheduler mq-deadline registered
[    1.878354] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    1.879737] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    1.880911] ACPI: button: Power Button [PWRF]
[    1.882410] ACPI: \_SB_.GSIG: Enabled at IRQ 22
[    1.884239] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    1.885614] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    1.888163] Linux agpgart interface v0.103
req access
sts 1 read_back val 1
req access
sts 1 read_back val 1
cmd_size 11
--->Got command TPM_CC_SelfTest
KVM: entry failed, hardware error 0xffffffff
EAX=0000000a EBX=00000000 ECX=00000000 EDX=00000000
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=00000000 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 00000000 00000000
CS =0000 00000000 00000000 00000000
SS =0000 00000000 00000000 00000000
DS =0000 00000000 00000000 00000000
FS =0000 00000000 00000000 00000000
GS =0000 00000000 00000000 00000000
LDT=0000 00000000 00000000 00000000
TR =0000 00000000 00000000 00000000
GDT=     00000000 00000000
IDT=     00000000 00000000
CR0=80050033 CR2=00000000 CR3=00000000 CR4=003506e0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000901
Code=<??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

[cclaudio]

@dbuono added a spinlock at the beginning of the SVSM handle_vtpm_request() function, however that did not solve the issue.