Skip to content

Add support for Intel Trust Domain Extensions (TDX) #228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 40 commits into from
Closed

Add support for Intel Trust Domain Extensions (TDX) #228

wants to merge 40 commits into from

Conversation

jakecorrenti
Copy link
Member

@jakecorrenti jakecorrenti commented Oct 21, 2024 •
edited
Loading

This PR adds support for the Intel Trust Domain Extensions (TDX) Confidential Computing architecture.

This is currently a draft as the following issues are present:

  • The guest is failing to complete the boot sequence. I suspect this is due to firmware issues, such as a lack of proper IDT setup and #VE handling
  • https://www.github.com/virtee/tdx needs to get published to crates.io before this can get merged

Before merging there are some commits that will be squashed and/or re-ordered.

There is also additional functionality that I would like to add such as:

  • Comprehensive CPUID configuration based off of the TDX capabilities reported by KVM_TDX_CAPABILITIES
  • Handle the following VMCALLs
    • TDG.VP.VMCALL<SetupEventNotifyInterrupt>
    • TDG.VP.VMCALL<GetQuote>
    • TDG.VP.VMCALL<MapGPA>
    • TDG.VP.VMCALL<REPORT_FATAL_ERROR>
  • Validate TDX Attributes when reported by KVM_TDX_CAPABILITIES
  • Update README.md and other docs
  • Make sure guests work with varying memory and vCPU configurations

Any early reviews are welcome.

@jakecorrenti jakecorrenti mentioned this pull request Oct 21, 2024
Open
@jakecorrenti
Copy link
Member Author

Temporarily pushing a mess of commits so that it can get cleaned up...

@jakecorrenti
Add TDX Makefile target
66b7130 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Add TDX Makefile target for examples
3e50bea 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add tdx-config-noattest.json
4af1d46 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
update dependencies
3f20d2b 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
update pre-existing CC feature checks
48198ae 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
temporarily link libkrun-tdx with libkrunfw-sev
02c0189 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Use kvm_userspace_memory_region2 and friends
4e7d38c 
In `memory_init` we need to use `kvm_userspace_memory_region2`,
`kvm_create_guest_memfd`, and `kvm_memory_attributes` for the TDX
architecture, otherwise it will fail.

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Disable KVM_CREATE_IRQCHIP for TDX
5eed8f1 
TDX does not use the `KVM_CREATE_IRQCHIP` ioctl, rather it enables the
`KVM_SPLIT_IRQCHIP` capability, which is handled by virtee/tdx.

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Disable register setup for TDX
68d1e25 
Registers are confidential for TDX, so configuring them through the KVM
API is not allowed.

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Implement Vm::new() for TDX
25b3a62 
Adds a new `inteltdx` module and implements a feature-flagged `new`
method for `VM to create a VM with the TDX architecure.

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
adjust params header
4adca37 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
change reset vector for TDX
3044cf5 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Implement tdx_secure_virt_prepare
8c65fbd 
Implements the `tdx_secure_virt_prepare` method which
in turn calls the `KVM_TDX_INIT_VM` TDX ioctl which does VM specific
initialization.

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
Change measured regions for TDX
fd8d51e 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
update how qboot is written to memory
3a3ac1c 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
update CPUID for vcpu on TDX
6f28cfc 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
init tdx vcpu
0b0a24c 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
init memory regions for tdx
9acdc26 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
finalize vm for tdx
657a0a3 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add serial device to TDX or else it hangs
fe1dadf 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
introduce support for TDX vCPU exit reason
6368123 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
implement TDG_VP_VMCALL_REPORT_FATAL_ERROR handler
7e7acfa 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT handler
77caaf5 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
keep track of guest memfd regions in vmm
5418aec 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add Vmm::vm_fd()
2b05f41 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add convert_memory()
6686e41 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
setup thread that when sent a GPA, length, and to_private, will conve…
311e0fd 
…rt the memory page accordingly

Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
add TDG_VP_VMCALL_MAP_GPA handler
8d02d59 
Signed-off-by: Jake Correnti <[email protected]>
@jakecorrenti
implement VcpuExit::MemoryFault
f8b3c5d 
Signed-off-by: Jake Correnti <[email protected]>
jakecorrenti and others added 11 commits March 6, 2025 09:00
@jakecorrenti
misc intermediate commit before adding IOAPIC
8e8c20e 
Signed-off-by: Jake Correnti <[email protected]>
@slp @jakecorrenti
libkrun: load libkrunfw as a dynamic library
c1f34bc 
Instead of linking statically against libkrunfw, load it as a dynamic
library. This will enable us to make its presence optional, which will
come handy when we support loading external payloads.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
libkrun: support external kernels on aarch64
3448d30 
Introduce initial support for external kernels by dealing with the
easier case: raw images on aarch64.

This commit adds a new function, "krun_set_kernel", which receives a
path to the external kernel.

Future commits will add support for more image formats and x86_64.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
Support multiple kernel formats
4db1e64 
In the previous commit we added support for the simplest type of
external kernel, a raw image that can be directly copied into the
VM's memory.

This commit builds on that to add support for multiple kernel
formats. The ones currently implemented are:

 - ELF: A kernel binary in ELF format (vmlinux).
 - PeGz: A PE binary embedding a kernel image compressed with GZIP.
 - ImageBz2: An Image file embedding a kernel compressed with BZIP2.
 - ImageGz: An Image file embedding a kernel compressed with GZIP.
 - ImageZstd: An Image file embedding a kernel compressed with ZSTD.

Adding new kernel formats should be quite straightforward.

Please note this change doesn't implement support for loading an
external initramfs. The main reason is that we can't guarantee to
maintain the control of the VM boot when using an arbitrary
initramfs.

This means that the external kernel must be built with, at least,
the following driver built-in:

- virtio-mmio
- virtio-console
- virtio-fs

Depending on the use case, more drivers might be required.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
Support external initramfs and custom cmdline
4c71406 
Complement the external kernel support added in the previous commits
with support for external initramfs and a custom kernel command line.

We don't have an immediate use case for this, as loading an external
initramfs may imply losing control of what's happening in the
guest, but it's a low-hanging fruit and may be useful for debugging or
a future use case.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
examples: add an example for external kernels
fb4aa04 
Add an example for loading an external kernel, initramfs and
custom kernel command line.

Signed-off-by: Sergio Lopez <[email protected]>
@jakecorrenti
misc commit for changing cfg feat. should be added as its own name
0aaea80 
Signed-off-by: Jake Correnti <[email protected]>
@slp @jakecorrenti
devices: generalize interrupt controllers
123a09b 
So far we didn't have a general way for creating an managing IRQ chips.
On KVM systems, the in-kernel chip was implicitly created by vstate,
while on HVF the userspace gicv3 was created explicitly.

In this change we generalize IRQ management behind IrqChip, and we  make
the creation of the IRQ chip explicit for every platform.

This is a pretty large commit, but all these changes must be done in an
atomic way.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
macos/vstate: unify MPIDR generation
045a0fd 
With HVF, we generate MPIDR by taking Vcpu.id and setting Aff1 to
that value by doing a left-shift, as this is what the GIC expects
to be found in the distributor.

But, in vstate::Vcpu::configure_aarch64, we were using the Vcpu.id
without left-shifting it.

For consistency, let's generate MPIDR for the Vcpu once and use it
everywhere.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
hvf: update Hypervisor.framework bindings
0049b2b 
Re-generate them from SDK 15.0.

Signed-off-by: Sergio Lopez <[email protected]>
@slp @jakecorrenti
devices: implement support for in-kernel HVF GICv3
f796d4d 
macOS 15 extended Hypervisor.framework with an in-kernel HVF GICv3
device. Using it reduces the cost of GIC operations and enables us to
use nested virt.

Add support for it and enable it automatically when the functions are
found in HVF.

Signed-off-by: Sergio Lopez <[email protected]>
@jakecorrenti
Copy link
Member Author

closing in favor of #313

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MatiasVara MatiasVara Awaiting requested review from MatiasVara MatiasVara will be requested when the pull request is marked ready for review MatiasVara is a code owner

@mtjhrc mtjhrc Awaiting requested review from mtjhrc mtjhrc will be requested when the pull request is marked ready for review mtjhrc is a code owner

@slp slp Awaiting requested review from slp slp will be requested when the pull request is marked ready for review slp is a code owner

@tylerfanelli tylerfanelli Awaiting requested review from tylerfanelli tylerfanelli will be requested when the pull request is marked ready for review tylerfanelli is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jakecorrenti @slp