Skip to content

KVM: Option to deploy a VM with existing volume/snapshot #10503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

KVM: Option to deploy a VM with existing volume/snapshot #10503

wants to merge 6 commits into from

Conversation

slavkap
Copy link
Contributor

@slavkap slavkap commented Mar 5, 2025

Description

This PR allows the users to deploy virtual machines with an existing volume or a snapshot.
For now it is limited only for KVM hypervisors and Zone-wide storage pools

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale

  • Major
  • Minor

How Has This Been Tested?

Environment
OS - Rocky Linux 8.9
2 Zones with one agent host on each
hypervisor KVM

Tested manually and with smoke tests with StorPool and NFS as a primary storage

@codecov Codecov
Copy link

codecov bot commented Mar 5, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 16.66667% with 155 lines in your changes missing coverage. Please review.

Project coverage is 3.93%. Comparing base (2d669db) to head (d8b4872).
Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java 36.48% 39 Missing and 8 partials ⚠️
...tastore/driver/StorPoolPrimaryDataStoreDriver.java 0.00% 44 Missing ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java 0.00% 37 Missing ⚠️
...he/cloudstack/api/command/user/vm/DeployVMCmd.java 27.27% 8 Missing ⚠️
...dstack/engine/orchestration/CloudOrchestrator.java 0.00% 7 Missing ⚠️
...n/java/com/cloud/vm/VirtualMachineManagerImpl.java 0.00% 5 Missing ⚠️
...a/com/cloud/network/lb/LoadBalanceRuleHandler.java 0.00% 1 Missing ⚠️
...etwork/contrail/management/ServiceManagerImpl.java 0.00% 1 Missing ⚠️
...a/com/cloud/network/vm/NetScalerVMManagerImpl.java 0.00% 1 Missing ⚠️
...om/cloud/consoleproxy/ConsoleProxyManagerImpl.java 0.00% 1 Missing ⚠️
... and 3 more
Additional details and impacted files 
@@              Coverage Diff              @@
##               main   #10503       +/-   ##
=============================================
- Coverage     16.60%    3.93%   -12.68%     
=============================================
  Files          5730      412     -5318     
  Lines        508082    33447   -474635     
  Branches      61770     6020    -55750     
=============================================
- Hits          84388     1317    -83071     
+ Misses       414259    31975   -382284     
+ Partials       9435      155     -9280
Flag Coverage Δ
uitests 3.93% <ø> (ø)
unittests 17.49% <16.66%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bernardodemarco bernardodemarco added this to the 4.21.0 milestone Mar 5, 2025
@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12681

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-12598)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 63251 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12598-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 1520.36 test_network.py
test_01_deploy_vm_with_existing_volume Error 7.34 test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot Error 0.06 test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template Failure 861.05 test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template Failure 862.76 test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume Failure 847.20 test_vm_lifecycle_with_snapshot_or_volume.py

@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12727

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-12645)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 61921 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12645-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 1520.50 test_network.py
test_01_deploy_vm_with_existing_volume Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py

@DaanHoogland
Copy link
Contributor

@slavkap the tests fail because the self doesn't have a snapshot that was supposed to be created in the class setup method. Is this a timing issue?

@slavkap
Copy link
Contributor Author

slavkap commented Mar 12, 2025

@blueorangutan package

@blueorangutan
Copy link

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12756

@slavkap
Copy link
Contributor Author

slavkap commented Jun 10, 2025

Thank you, @DaanHoogland and @nvazquez, for the reviews! I've addressed your comments in the latest commit.
@rp, thank you for testing this! I've added the option to set the required VM tags on the volume through VolumeOrchestrator::provideVmInfoToTheStorageVolume. You can check this if you need it for the Linstor plug-in

@slavkap
Copy link
Contributor Author

slavkap commented Jun 11, 2025

@blueorangutan package

@blueorangutan
Copy link

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13711

@github-actions GitHub Actions
Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

slavkap added 5 commits June 12, 2025 10:27
@slavkap
Option to deploy a VM with existing volume/snapshot
ff79b5e
@slavkap
smoke test changes
e96e841 
check if the hypervisor is KVM
check if the primary storage's scope is ZONE wide
@slavkap
skip all tests if the storage isn't Zone-Wide and the hypervisor isn'…
44e0c73 
…t KVM
@slavkap
support StorPool tags
be5b5bd 
add StorPool tags to a volume created from snapshot or to a volume which
will be attached as a ROOT to a new VM
@slavkap
Add StorPool tags on the new ROOT volume
06428dd 
Add the StorPool's tags when volume is created from a snapshot or a
volume is attached as a ROOT to a VM
Addressed reviews
@slavkap slavkap force-pushed the deploy-vm-with-existing-volume-or-snapshot branch from b919361 to 06428dd Compare June 13, 2025 07:29
@nvazquez nvazquez closed this Jun 13, 2025
@github-project-automation github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.21.0 Jun 13, 2025
@nvazquez nvazquez reopened this Jun 13, 2025
@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13768

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-13523)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 53502 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t13523-kvm-ol8.zip
Smoke tests completed. 142 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

DaanHoogland
DaanHoogland reviewed Jun 16, 2025
View reviewed changes
@slavkap
addressed review
d8b4872 
Added new line
@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13790

@github-actions GitHub Actions
Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DaanHoogland DaanHoogland DaanHoogland left review comments

@nvazquez nvazquez nvazquez left review comments

@bernardodemarco bernardodemarco Awaiting requested review from bernardodemarco

@rajujith rajujith Awaiting requested review from rajujith

Assignees
No one assigned
Labels
component:api component:compute component:integration-test component:kubernetes component:marvin component:networking component:orchestration component:primary-storage component:secondary-storage component:ssvm component:storage component:vpc Python Warning... Python code Ahead! status:has-conflicts status:needs-testing
Projects
Apache CloudStack 4.21.0
Status: Done
Milestone
4.21.0
Development

Successfully merging this pull request may close these issues.

Streamlined Instance Cloning
7 participants
@slavkap @DaanHoogland @blueorangutan @rp- @weizhouapache @nvazquez @bernardodemarco