Add support for arm cca #211

MatiasVara · 2024-07-29T15:46:06Z

This PR aims at adding support to build realm guests. First commit adds support for create_guest_memfd() and set_user_memory_region2(). To do this, the memory_init() is modified by adding a boolean parameter. This is required when building a confidential guest for arm cca and probably also required by other cases.
The second commit imports the virtee/cca crate and adds the steps to build a cca guest. The following items should be completed before merge the PR:

use populate() only for kernel and initialize() for non-kernel area
handle the case in which the guest switch from shared -> private
correctly calculation of max_ipa

This has been testing using the v7 series for Linux as a guest and v6 series for KVM on FVP model.

Feedback is welcome.

jakecorrenti · 2025-01-09T21:33:07Z

@MatiasVara is this ready for review, or should I wait?

tylerfanelli · 2025-01-09T21:40:26Z

@jakecorrenti I'm addressing the KVM guest_memfd changes (which @MatiasVara also added in this PR) to my latest SEV-SNP patches. This will probably require a rebase after that. Once that happens, it will likely be ready for a review.

MatiasVara · 2025-01-10T09:34:36Z

@jakecorrenti I'm addressing the KVM guest_memfd changes (which @MatiasVara also added in this PR) to my latest SEV-SNP patches. This will probably require a rebase after that. Once that happens, it will likely be ready for a review.

I think the PR is ready to review. The current issue is that it does not work for the latest series for KVM (v6). I am investigating the issue.

MatiasVara · 2025-02-12T10:59:30Z

I just updated the PR with the fix for v6 series. The fix is only to set the KVM_ARM_VCPU_REC feature during the initialization of VCPUs otherwise vcpu_finalize would fail.

MatiasVara · 2025-02-12T11:39:58Z

@jakecorrenti @tylerfanelli @slp Feel free to give a quick look. I am pretty sure there many things to fix.

Signed-off-by: Matias Ezequiel Vara Larsen <[email protected]>

Enable to build confidential guests using ARM CCA (Confidential Computing Architecture). This work relies on v7 series for Linux and v5 series for KVM. This has been tested only on the corresponding FVP model simulator. For testing, you require specific kvm-ioctls and kvm-bindings crates. Signed-off-by: Matias Ezequiel Vara Larsen <[email protected]>

jakecorrenti

I didn't actually run it, just looked at the code. Overall looks good. Few comments, though.

Thanks for the work!

jakecorrenti · 2025-02-19T17:42:16Z

src/arch/Cargo.toml

+default = ["cca"]
+cca = []


We shouldn't make cca the default here. IMO, it should follow suit with the others

Suggested change

default = ["cca"]

cca = []

cca = ["tee"]

If I understand correctly, when I compile with cca flag:

cargo build --release --features cca

The tee feature would be enabled. However, most of the code for tee is not required and I ended up doing this everywhere:

- #[cfg(feature = "tee")] + #[cfg(all(feature = "tee", not(feature = "cca")))] let qboot_bundle = vm_resources .qboot_bundle() .ok_or(StartMicrovmError::MissingKernelConfig)?;

Can't cca be just a different feature?

jakecorrenti · 2025-02-19T17:43:46Z

src/arch/Cargo.toml

+kvm-bindings = { version = ">=0.8", features = ["fam-wrappers"] , git = "https://github.com/virtee/kvm-bindings", branch = "add_bindings_for_realms" }
+kvm-ioctls = { version = ">=0.17", git = "https://github.com/virtee/kvm-ioctls", branch = "cca" }


Can you make a PR once these branches are ready? I'd like to have the deps be on the main branch

Do you mean to use the main branch rather than the cca?

Exactly. I'm going to have to make some changes for my TDX work, so I think we'll need to merge your CCA work and my TDX work into the main branch and then use that here.

@tylerfanelli Do you think we can merge the cca branches into main for kvm-ioctl/bindings repos? Or, shall we have one branch per flavor? Bear in mind that those repo are temporal until changes are upstreamed.

jakecorrenti · 2025-02-19T17:44:31Z

src/cpuid/Cargo.toml

+kvm-bindings = { version = ">=0.8", features = ["fam-wrappers"] , git = "https://github.com/virtee/kvm-bindings", branch = "add_bindings_for_realms" }
+kvm-ioctls = { version = ">=0.17", git = "https://github.com/virtee/kvm-ioctls", branch = "cca" }


Same comment here

jakecorrenti · 2025-02-19T17:44:53Z

src/devices/Cargo.toml

+default = ["cca"]
 tee = []
+cca = []


Suggested change

default = ["cca"]

tee = []

cca = []

cca = ["tee"]

Should I not leave tee =[] here?

You should. I think that was a suggestion error. I was trying to suggest just getting rid of default = ...

jakecorrenti · 2025-02-24T20:18:52Z

src/libkrun/Cargo.toml

 crossbeam-channel = "0.5"
 env_logger = "0.9.0"
 libc = ">=0.2.39"
 log = "0.4.0"
 once_cell = "1.4.1"

+kvm-bindings = { version = ">=0.8", features = ["fam-wrappers"] , git = "https://github.com/virtee/kvm-bindings", branch = "add_bindings_for_realms" }


same comment here

jakecorrenti · 2025-02-24T21:04:55Z

src/vmm/src/builder.rs

+        //vmm.kernel_cmdline.insert_str("tsi_hijack")?;
        #[cfg(feature = "net")]


Should this be CCA only? There's another one a few lines below as well

I can't remember why I commented this line.

src/vmm/src/linux/vstate.rs

jakecorrenti · 2025-02-24T21:14:08Z

src/vmm/src/linux/vstate.rs

+                // TODO: to remove this
+                Unsupported(39) => Ok(VcpuEmulation::Handled),


leftover TODO

jakecorrenti · 2025-02-24T21:15:31Z

src/vmm/src/linux/vstate.rs

+        id: u8,
+        vm_fd: &VmFd,
+        exit_evt: EventFd,
+        sender_io: Sender<MemProperties>,


should be CCA only

The memory fault exception isn't TEE-specific (although it could be used that way), so I don't think I agree that this should be CCA only.

You're right. I did this part of the review before you and I had talked about the order of patches.

jakecorrenti · 2025-02-24T21:16:32Z

src/vmm/src/builder.rs

@@ -1103,17 +1231,19 @@ fn create_vcpus_aarch64(
    guest_mem: &GuestMemoryMmap,
    entry_addr: GuestAddress,
    exit_evt: &EventFd,
+    sender_io: Sender<MemProperties>,


should be CCA only

MatiasVara force-pushed the use-guest-memfd branch from 702d17f to 86f75cd Compare July 29, 2024 16:04

MatiasVara force-pushed the use-guest-memfd branch from b47ef94 to 9e6e5bb Compare August 14, 2024 08:09

MatiasVara force-pushed the use-guest-memfd branch from 9e6e5bb to 68dff0c Compare August 22, 2024 15:20

MatiasVara changed the title ~~Enable creating guest memory regions with create_guest_memfd() and set_user_memory_region2()~~ Add support for arm cca Aug 26, 2024

MatiasVara force-pushed the use-guest-memfd branch 2 times, most recently from d5952be to d4362d6 Compare September 4, 2024 12:10

MatiasVara force-pushed the use-guest-memfd branch 3 times, most recently from cae99a3 to 4a0e61b Compare September 10, 2024 11:40

MatiasVara force-pushed the use-guest-memfd branch 3 times, most recently from 5089b4f to e7e2a90 Compare September 13, 2024 17:30

MatiasVara force-pushed the use-guest-memfd branch 3 times, most recently from dae13da to 6cebf95 Compare October 1, 2024 10:44

MatiasVara force-pushed the use-guest-memfd branch from 6cebf95 to 94688b2 Compare October 24, 2024 09:46

MatiasVara force-pushed the use-guest-memfd branch from 94688b2 to 2077cfc Compare November 21, 2024 11:41

MatiasVara force-pushed the use-guest-memfd branch 2 times, most recently from 18e9a7d to edb2f4a Compare December 19, 2024 13:58

MatiasVara marked this pull request as ready for review December 19, 2024 15:05

MatiasVara requested review from mtjhrc, slp and tylerfanelli as code owners December 19, 2024 15:05

MatiasVara force-pushed the use-guest-memfd branch from edb2f4a to 5ba47b6 Compare December 19, 2024 15:59

MatiasVara force-pushed the use-guest-memfd branch 2 times, most recently from ba8a4b1 to 0cf214d Compare February 12, 2025 10:53

MatiasVara force-pushed the use-guest-memfd branch from 0cf214d to 866514c Compare February 12, 2025 11:03

MatiasVara force-pushed the use-guest-memfd branch from 866514c to 3d4903a Compare February 12, 2025 12:46

MatiasVara force-pushed the use-guest-memfd branch from 3d4903a to 7b1f763 Compare February 20, 2025 11:25

MatiasVara added 2 commits February 20, 2025 07:30

Use create_guest_memfd() and set_user_memory_region2()

8affa13

Signed-off-by: Matias Ezequiel Vara Larsen <[email protected]>

MatiasVara force-pushed the use-guest-memfd branch from 7b1f763 to 8b97dc4 Compare February 20, 2025 12:30

jakecorrenti suggested changes Feb 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for arm cca #211

Add support for arm cca #211

MatiasVara commented Jul 29, 2024 •

edited

Loading

jakecorrenti commented Jan 9, 2025

tylerfanelli commented Jan 9, 2025 •

edited

Loading

MatiasVara commented Jan 10, 2025

MatiasVara commented Feb 12, 2025

MatiasVara commented Feb 12, 2025

jakecorrenti left a comment

jakecorrenti Feb 19, 2025

MatiasVara Mar 4, 2025 •

edited

Loading

jakecorrenti Feb 19, 2025

MatiasVara Feb 25, 2025

jakecorrenti Feb 25, 2025

MatiasVara Feb 26, 2025

jakecorrenti Feb 19, 2025

jakecorrenti Feb 19, 2025

MatiasVara Feb 27, 2025

jakecorrenti Feb 27, 2025

jakecorrenti Feb 24, 2025

jakecorrenti Feb 24, 2025

MatiasVara Feb 25, 2025

jakecorrenti Feb 24, 2025

jakecorrenti Feb 24, 2025

tylerfanelli Feb 26, 2025

jakecorrenti Feb 26, 2025

jakecorrenti Feb 24, 2025

		kvm-bindings = { version = ">=0.8", features = ["fam-wrappers"] , git = "https://github.com/virtee/kvm-bindings", branch = "add_bindings_for_realms" }
		kvm-ioctls = { version = ">=0.17", git = "https://github.com/virtee/kvm-ioctls", branch = "cca" }

		//vmm.kernel_cmdline.insert_str("tsi_hijack")?;
		#[cfg(feature = "net")]

		// TODO: to remove this
		Unsupported(39) => Ok(VcpuEmulation::Handled),

Add support for arm cca #211

Are you sure you want to change the base?

Add support for arm cca #211

Conversation

MatiasVara commented Jul 29, 2024 • edited Loading

jakecorrenti commented Jan 9, 2025

tylerfanelli commented Jan 9, 2025 • edited Loading

MatiasVara commented Jan 10, 2025

MatiasVara commented Feb 12, 2025

MatiasVara commented Feb 12, 2025

jakecorrenti left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MatiasVara Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MatiasVara commented Jul 29, 2024 •

edited

Loading

tylerfanelli commented Jan 9, 2025 •

edited

Loading

MatiasVara Mar 4, 2025 •

edited

Loading