Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

providers/libvirt: add support for aarch64 host #2193

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

xutao323
Copy link
Contributor

@xutao323 xutao323 commented Dec 6, 2024

Create domainXML for aarch64/arm64 arch in cloud-provider/libvirt, and check iommu=on for virtio device only in test.

This patch is similar to 63a002c, and ARM CCA CVM will be added later like 4422f14, when it's ready. This will initially support non-CCA workloads and eventually allow the creation of a Realm on a local/remote host.

go test on aarch64/arm64 host:

$ LIBVIRT_URI="qemu+ssh://[email protected]/system?no_verify=1" go test -v
=== RUN   TestCloudInit
temp file: /tmp/CloudInit-12504980.iso--- PASS: TestCloudInit (0.01s)
=== RUN   TestInMemoryCopier
--- PASS: TestInMemoryCopier (0.00s)
=== RUN   TestLibvirtConnection
2024/12/06 11:27:46 [adaptor/cloud/libvirt] Created libvirt connection
--- PASS: TestLibvirtConnection (0.64s)
=== RUN   TestGetArchitecture
2024/12/06 11:27:47 [adaptor/cloud/libvirt] Created libvirt connection
--- PASS: TestGetArchitecture (0.63s)
=== RUN   TestCreateDomainXMLs390x
2024/12/06 11:27:47 [adaptor/cloud/libvirt] Created libvirt connection
    libvirt_test.go:110: Skipping because architecture is [aarch64] and not [s390x].
--- SKIP: TestCreateDomainXMLs390x (0.61s)
=== RUN   TestCreateDomainXMLaarch64
2024/12/06 11:27:48 [adaptor/cloud/libvirt] Created libvirt connection
--- PASS: TestCreateDomainXMLaarch64 (0.60s)
PASS
ok  	github.com/confidential-containers/cloud-api-adaptor/src/cloud-providers/libvirt	2.590s

go test on x86_64/amd64 host:

$ LIBVIRT_URI="qemu+ssh://[email protected]/system?no_verify=1" go test -v
=== RUN   TestCloudInit
temp file: /tmp/CloudInit-2407912272.iso--- PASS: TestCloudInit (0.00s)
=== RUN   TestInMemoryCopier
--- PASS: TestInMemoryCopier (0.00s)
=== RUN   TestLibvirtConnection
2024/12/06 11:34:11 [adaptor/cloud/libvirt] Created libvirt connection
--- PASS: TestLibvirtConnection (0.60s)
=== RUN   TestGetArchitecture
2024/12/06 11:34:12 [adaptor/cloud/libvirt] Created libvirt connection
--- PASS: TestGetArchitecture (0.36s)
=== RUN   TestCreateDomainXMLs390x
2024/12/06 11:34:12 [adaptor/cloud/libvirt] Created libvirt connection
    libvirt_test.go:110: Skipping because architecture is [x86_64] and not [s390x].
--- SKIP: TestCreateDomainXMLs390x (0.28s)
=== RUN   TestCreateDomainXMLaarch64
2024/12/06 11:34:12 [adaptor/cloud/libvirt] Created libvirt connection
    libvirt_test.go:147: Skipping because architecture is [x86_64] and not [aarch64].
--- SKIP: TestCreateDomainXMLaarch64 (0.29s)
PASS
ok  	github.com/confidential-containers/cloud-api-adaptor/src/cloud-providers/libvirt	1.541s

libvirt E2E test needs additional changes in build scripts (operator and cloud-api-adaptor to build the new code) plus podvm to run on aarch64/arm64 machine. Those will be separated as they're more related to build.

E2E tests were done on aarch64/arm64 baremetal server to run demo peer-pods:
(based on v0.9.0 and v0.10.0 podvm, latest v0.11.0 podvm introduced cached artifacts in #2074).

$ kubectl get pods -o wide
NAME             READY   STATUS    RESTARTS   AGE     IP           NODE                 NOMINATED NODE   READINESS GATES
busybox-remote   1/1     Running   0          6m16s   10.244.1.8   peer-pods-worker-0   <none>           <none>

$ kubectl exec pods/busybox-remote -- uname -srim
Linux 5.4.0-136-generic aarch64 unknown

$ kcli list vm
+-------------------------------+--------+-----------------+------------+-----------+---------+
|              Name             | Status |        Ip       |   Source   |    Plan   | Profile |
+-------------------------------+--------+-----------------+------------+-----------+---------+
|      peer-pods-ctlplane-0     |   up   |  192.168.122.20 | ubuntu2204 | peer-pods |  kvirt  |
|       peer-pods-worker-0      |   up   | 192.168.122.199 | ubuntu2204 | peer-pods |  kvirt  |
| podvm-busybox-remote-62291ece |   up   | 192.168.122.237 |            |           |         |
+-------------------------------+--------+-----------------+------------+-----------+---------+

$ virsh list
 Id   Name                            State
-----------------------------------------------
 70   peer-pods-ctlplane-0            running
 71   peer-pods-worker-0              running
 72   podvm-busybox-remote-62291ece   running 

@xutao323 xutao323 requested a review from a team as a code owner December 6, 2024 11:44
@stevenhorsman stevenhorsman added the test_e2e_libvirt Run Libvirt e2e tests label Dec 6, 2024
create domainXML for aarch64/arm64 architecture in cloud-provider/libvirt,
and check iommu=on for virtio device only in test.

Signed-off-by: Tao Xu <[email protected]>
Reviewed-by: Seunguk Shin <[email protected]>
Reviewed-by: Nick Connolly <[email protected]>
Copy link
Member

@stevenhorsman stevenhorsman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is probably an opportunity to refactor the domainXML creation functions so there is less duplicated code, but I think that could be done as a later step, if necessary

@xutao323
Copy link
Contributor Author

xutao323 commented Dec 9, 2024

Thanks for the review. Agreed that domainXML creation can be refactored for those common configs, and it can be done separately before adding new CVM configs.

For the test failures, gofmt -s from golangci-lint is fixed. Latest tests hit #2179 and it shall be irrelevant.

Copy link
Member

@bpradipt bpradipt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@bpradipt bpradipt merged commit aee18e6 into confidential-containers:main Dec 10, 2024
39 of 40 checks passed
@xutao323 xutao323 deleted the libvirt_aarch64 branch December 10, 2024 11:00
@xutao323
Copy link
Contributor Author

Thanks @stevenhorsman @bpradipt for reviewing the changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test_e2e_libvirt Run Libvirt e2e tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants