Dornerworks x64 VM patch #16

nomadeel · 2021-03-26T04:49:51Z

This PR contains commits from Dornerworks' x64 VM patches for the VMM: https://github.com/dornerworks/camkes-vm/commits/open_source_release. Parts of Dornerworks' patches have been updated and/or changed slightly to match the newer VMM structure compared to what the patches originally targeted.

oliver-wm · 2021-04-09T05:07:18Z

Is this the one that was on bitbucket?

nomadeel · 2021-04-12T04:50:12Z

I think so, yeah.

EDIT: No, nevermind, it's not. This is new.

oliver-wm · 2021-04-14T23:54:02Z

I think so, yeah.

EDIT: No, nevermind, it's not. This is new.

Ok will review then

kent-mcleod · 2021-05-14T10:19:49Z

libsel4vm/src/arch/x86/vmcs.c

+#ifdef CONFIG_ARCH_X86_64
+    vm_vmcs_init_64_bit_guest(vcpu);
+#else
+    vm_vmcs_init_32_bit_guest(vcpu);


There are example apps that run a 32bit guest on a 64 bit host. Is this no longer something that we will support?

Yep, just found out as well when fixing some builds. I need to make extra changes to the various repositories to allow 32-bit guest on 64-bit host.

Have these changes been made?

libsel4vm/src/arch/x86/ept.c

chrisguikema · 2022-09-28T19:13:37Z

What needs to happen to get this merged?

axel-h · 2022-09-28T19:53:44Z

libsel4vm/src/arch/x86/guest_x86_context.c

@@ -4,6 +4,13 @@
 * SPDX-License-Identifier: BSD-2-Clause
 */

+/* Under release mode, this file will get built using -O3. Howerver on gcc 8.4,


Is this a problem specific to GCC 8.4 or does it happen with newer versions and clang also? Such issues leave an odd feeling that there is an issue with the code (maybe alignment?) that I consider worth investigating. However, it should not be seen as a blocker for merging this as it temporarily fixes a problem.

I have created #75 to track this.

axel-h · 2022-09-28T19:59:27Z

What needs to happen to get this merged?

Seems it needs a rebase now. I have a few comments on the code quality, but have to leave it to other to comment on the x64 changes.

libsel4vm/arch_include/x86/sel4vm/arch/vmcs_fields.h

libsel4vm/src/arch/x86/boot.c

axel-h · 2022-09-28T20:26:29Z

libsel4vm/src/arch/x86/processor/decode.c

@@ -368,15 +499,65 @@ uintptr_t vm_emulate_realmode(vm_vcpu_t *vcpu, uint8_t *instr_buf,
            case 0xa1:
                /* mov offset memory to eax */
                instr++;
+#ifdef CONFIG_ARCH_X86_64
+                memcpy(&mem, instr, 4);


When using sizeof(seL4_Word)/2 there is no need to duplicate the code. Might apply in other places also.

Suggested change

memcpy(&mem, instr, 4);

memcpy(&mem, instr, sizeof(seL4_Word)/2 );

libsel4vmmplatsupport/src/arch/x86/acpi.c

libsel4vm/src/arch/x86/processor/msr.c

abrandnewusername · 2023-05-02T09:52:44Z

Update:

Fixed most of the issues mentioned in previous comments
Changed the guard from CONFIG_ARCH_X86_64 to CONFIG_X86_64_VTX_64BIT_GUESTS due to a decision made in the related PR: Dornerworks x64 VM seL4#324 (comment)).

Future works: Current implementation for x64 support relies on the corresponding kernel implementation (see seL4/seL4#324), which does not support running both 32-bit VM guest and 64-bit VM guest on the same system. It will be ideal to implement this feature in the future.

wom-bat

A few minor changes can clean up some of the ifdefs. Otherwise, good to go.

This large commit combines a number of smaller commits in order to add 64-bit VM support to the VMM. The commits do the following: * Load elfs of the same architecture size * Support reading 64-bit vaddrs from elf files * Create initial address space based on guest architecture * Allow instruction decoding of 4-level paging scheme * Do not exit VMX state based on CR3 load/store in 64-bit mode * Use seL4_Word for vmcs and user context fields * Handle additional general purpose registers * Handle 64-bit MSRs * Configure 64-bit guests to boot in long mode * Let guest know about 64-bit hardware features in 64-bit mode * Add FADT and DSDT tables * Track guest state for additional 64-bit registers * Add new function to print 64-bit guest context * Properly emulate 64-bit trampoline code * Define access rights macros for vmcs initialization * Hardcode FADT table information * Set an initial stack pointer before running guest * Handle fetching cross-page instructions * Add additional x86 instruction prefixes CCDC-GVSC DISTRIBUTION A. Approved for public release; distribution unlimited. OPSEC#4481. Co-authored-by: Chris Guikema <[email protected]> Signed-off-by: Damon Lee <[email protected]>

Signed-off-by: Damon Lee <[email protected]>

As the comment in the file explains, optimisation level -O3 on gcc 8.4 is too aggressive and causes issues for the guest VM. The problem isn't limited to a single function or group of functions but rather the entire file for some reason that requires extra investigation (but is not worth the time to do so). Signed-off-by: Damon Lee <[email protected]>

Previously, the physical address of the ACPI tables were being tracked. This caused a Linux error while parsing the ACPI tables. Since Linux does not have access to those memory regions, they would appear empty, causing an Invalid Table Length bug print. By tracking the virtual address that Linux expects and placing the vaddr into the ACPI tables, Linux can parse the tables properly. Signed-off-by: Chris Guikema <[email protected]>

switch from CONFIG_ARCH_X86_64 to CONFIG_X86_64_VTX_64BIT_GUESTS to allow 32 bit guests on 64 bit hosts; fix print format for seL4_Word; misc improvement. Signed-off-by: Jingyao Zhou <[email protected]>

Signed-off-by: Jingyao Zhou <[email protected]>

nomadeel force-pushed the dornerworks_x64 branch from 58ea698 to f4a922d Compare March 26, 2021 04:51

nomadeel force-pushed the dornerworks_x64 branch from f4a922d to 0f9286d Compare April 9, 2021 04:49

nomadeel requested review from kent-mcleod, sylGauthier and szhuang April 9, 2021 04:57

nomadeel force-pushed the dornerworks_x64 branch from 7be172e to 6c21c8a Compare April 15, 2021 00:09

This was referenced May 12, 2021

Dornerworks x64 tiiuae/seL4_projects_libs#4

Merged

Dornerworks x64 VM patch tiiuae/seL4_projects_libs#5

Merged

kent-mcleod reviewed May 14, 2021

View reviewed changes

kent-mcleod approved these changes May 14, 2021

View reviewed changes

nomadeel mentioned this pull request Sep 12, 2021

Dornerworks x64 VM seL4/seL4#324

Merged

axel-h reviewed Jun 27, 2022

View reviewed changes

libsel4vm/src/arch/x86/ept.c Outdated Show resolved Hide resolved

axel-h mentioned this pull request Sep 28, 2022

Investigate VMM issues with GCC8.4 at "-o3" #75

Open

axel-h reviewed Sep 28, 2022

View reviewed changes

libsel4vm/arch_include/x86/sel4vm/arch/vmcs_fields.h Outdated Show resolved Hide resolved

axel-h reviewed Sep 28, 2022

View reviewed changes

libsel4vm/src/arch/x86/boot.c Outdated Show resolved Hide resolved

axel-h reviewed Sep 28, 2022

View reviewed changes

libsel4vmmplatsupport/src/arch/x86/acpi.c Outdated Show resolved Hide resolved

axel-h reviewed Sep 28, 2022

View reviewed changes

libsel4vm/src/arch/x86/processor/msr.c Outdated Show resolved Hide resolved

abrandnewusername self-assigned this Jan 17, 2023

abrandnewusername force-pushed the dornerworks_x64 branch from 6c21c8a to 0f67bde Compare May 2, 2023 05:25

abrandnewusername mentioned this pull request May 2, 2023

x86: Add new minimal_64 test seL4/camkes-vm-examples#40

Closed

abrandnewusername force-pushed the dornerworks_x64 branch 2 times, most recently from b86774f to ccb27a8 Compare May 2, 2023 09:51

abrandnewusername requested review from axel-h and wom-bat May 3, 2023 02:08

wom-bat approved these changes May 3, 2023

View reviewed changes

Damon Lee and others added 6 commits June 6, 2023 16:58

trivial: Fix compiler warnings

0703b94

Signed-off-by: Damon Lee <[email protected]>

64-bit-vms: code refactoring

8227e00

switch from CONFIG_ARCH_X86_64 to CONFIG_X86_64_VTX_64BIT_GUESTS to allow 32 bit guests on 64 bit hosts; fix print format for seL4_Word; misc improvement. Signed-off-by: Jingyao Zhou <[email protected]>

trivial: remove unused variable for acpi.c

dd13a24

Signed-off-by: Jingyao Zhou <[email protected]>

abrandnewusername force-pushed the dornerworks_x64 branch from ccb27a8 to dd13a24 Compare June 6, 2023 06:58

abrandnewusername merged commit d96def8 into seL4:master Jun 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dornerworks x64 VM patch #16

Dornerworks x64 VM patch #16

nomadeel commented Mar 26, 2021

oliver-wm commented Apr 9, 2021

nomadeel commented Apr 12, 2021 •

edited

Loading

oliver-wm commented Apr 14, 2021

kent-mcleod May 14, 2021

nomadeel Jul 8, 2021

wom-bat Apr 13, 2022

chrisguikema commented Sep 28, 2022

axel-h Sep 28, 2022

axel-h commented Sep 28, 2022

axel-h Sep 28, 2022

abrandnewusername commented May 2, 2023 •

edited

Loading

wom-bat left a comment

	memcpy(&mem, instr, 4);
	memcpy(&mem, instr, sizeof(seL4_Word)/2 );

Dornerworks x64 VM patch #16

Dornerworks x64 VM patch #16

Conversation

nomadeel commented Mar 26, 2021

oliver-wm commented Apr 9, 2021

nomadeel commented Apr 12, 2021 • edited Loading

oliver-wm commented Apr 14, 2021

kent-mcleod May 14, 2021

Choose a reason for hiding this comment

nomadeel Jul 8, 2021

Choose a reason for hiding this comment

wom-bat Apr 13, 2022

Choose a reason for hiding this comment

chrisguikema commented Sep 28, 2022

axel-h Sep 28, 2022

Choose a reason for hiding this comment

axel-h commented Sep 28, 2022

axel-h Sep 28, 2022

Choose a reason for hiding this comment

abrandnewusername commented May 2, 2023 • edited Loading

wom-bat left a comment

Choose a reason for hiding this comment

nomadeel commented Apr 12, 2021 •

edited

Loading

abrandnewusername commented May 2, 2023 •

edited

Loading