Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] "waagent -deprovision+user" does not remove /var/lib/hyperv/, resulting in stale kvp_pool files in a new VM #3276

Open
dcui opened this issue Dec 13, 2024 · 1 comment
Assignees
Labels

Comments

@dcui
Copy link

dcui commented Dec 13, 2024

When preparing a Linux VHD image for Azure, we usually run "sudo waagent -deprovision+user" (see https://learn.microsoft.com/en-us/azure/virtual-machines/generalize) and assume that the VHD won't contain any stale info from the previous VM.

It turns out this assumption is not true because walinuxagent doesn't remove the folder /var/lib/hyperv/, resulting in stale kvp_pool files in a new VM:
/var/lib/hyperv/.kvp_pool_0 /var/lib/hyperv/.kvp_pool_1 /var/lib/hyperv/.kvp_pool_2 /var/lib/hyperv/.kvp_pool_3 /var/lib/hyperv/.kvp_pool_4

Some of the pool files may contain stale and incorrect info, e.g. /var/lib/hyperv/.kvp_pool_3 contains the Windows host name. It looks like under some condition the KVP daemon fails to update the host name in /var/lib/hyperv/.kvp_pool_3 before the VM's first reboot (the root cause is still unclear yet, but it's a different issue); when this happens, a user may be concerned as the host name may be used to implement some kind of load balancing, and an unexpected incorrect host name in the kvp pool file can mislead the load balancing code.

The request is: "waagent -deprovision+user" should remove /var/lib/hyperv/.

Steps to reproduce the behavior:

  1. Start a Linux VM of any Linux distro on Azure or a local Hyper-V host, e.g. Ubutnu/RHEL/SLES. Make sure the KVP daemon is running, and you see the files "ls /var/lib/hyperv/.kvp_pool_*".
  2. Run "sudo waagent -deprovision+user".
  3. "ls /var/lib/hyperv/.kvp_pool_*" still shows the kvp pool files.

This behavior exists in all versions of WALinuxAgent.

@maddieford
Copy link
Contributor

@longlimsft, I see rdma logic in the agent uses these kvp files to get driver info, etc:

driver_info_source = '/var/lib/hyperv/.kvp_pool_0'

Agent team doesn't have expertise on this usage, or the impact of removing this directory during deprovisioning.
Assigning this issue to RDMA team to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants