UM780 + Linux Guest Errors

[Snuupy]

Following #5, I am trying to passthrough the igpu to a linux guest. amdgpu_top works on host prior to starting the guide. amdgpu_top reports no devices in ubuntu vm. What am I doing wrong?

I set VRAM to 16GB and VM RAM to 16GB also and got some PSP error messages in journalctl. I increased VM RAM to 24GB (bigger than vram just in case), and created the vbios using the c script (since my cpu is a 7840HS, the non-XTX version)

I also tried setting VRAM to AUTO (1GB) and I still get these errors:

Here is my /etc/pve/qemu-server/102.conf:

args: -cpu 'host,-hypervisor,kvm=off'
boot: order=scsi0;net0
cores: 8
cpu: custom-7840HSigpupassthrough
hostpci0: 0000:c4:00.0,pcie=1,romfile=vbios_1002_15bf.bin
hostpci1: 0000:c4:00.1,pcie=1
machine: q35
memory: 24576
meta: creation-qemu=8.1.5,ctime=1710991650
name: ubuntu2310-server
net0: virtio=xxxxxxxxxxxxxxx,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-102-disk-0,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=xxxxxxxxxxxxxxxx
sockets: 1
unused0: local-lvm:vm-102-disk-1
vmgenid: xxxxxxxxxxxxxxxxxxxx

and lspci:

root@SnuUM780:~# lspci -nn | grep -e 'AMD/ATI'
c4:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c2)
c4:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]

root@SnuUM780:~# cat /etc/default/grub # has
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"

root@SnuUM780:~# cat /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

root@SnuUM780:~# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:15bf,1002:1640
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci

root@SnuUM780:~# lspci -nnk
c4:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c2)
        Subsystem: Device [1f4c:b002]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
c4:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]
        Subsystem: Device [1f4c:b002]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

I thought I was going crazy! But no - this does show up in windows:

[Snuupy]

update: added the .5 iommu group to proxmox passthrough, looks like it works on windows (but the webgl animation still shows 15fps with 100cpu util for some reason, don't know why)

I reinstalled the ubuntu vm and switched from legacy bios to UEFI to copy the windows setup, same error:

[    1.087956] shpchp 0000:05:01.0: pci_hp_register failed with error -16
[    1.087962] shpchp 0000:05:01.0: Slot initialization failed
...
[    4.764516] [drm:psp_v13_0_4_ring_create [amdgpu]] *ERROR* Failed to wait for trust OS ready for ring creation
[    4.765006] [drm:psp_hw_start [amdgpu]] *ERROR* PSP create ring failed!
[    4.952710] [drm:psp_v13_0_4_ring_destroy [amdgpu]] *ERROR* Fail to stop psp ring
[    4.953167] [drm:psp_hw_init [amdgpu]] *ERROR* PSP firmware loading failed
[    4.953907] [drm:amdgpu_device_fw_loading [amdgpu]] *ERROR* hw_init of IP block <psp> failed -22
[    4.954239] amdgpu 0000:01:00.0: amdgpu: amdgpu_device_ip_init failed
[    4.954436] amdgpu 0000:01:00.0: amdgpu: Fatal error during GPU init
[    4.954628] amdgpu 0000:01:00.0: amdgpu: amdgpu: finishing device.

and strangely:

root@ubuntu2310:~# lspci -nn
00:01.0 VGA compatible controller [0300]: Device [1234:1111] (rev 02)
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c2)
02:00.0 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]
03:00.0 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor [1022:15e2] (rev 63)

but the same lspci -nn on the host would be at a different address:

c4:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c2)
c4:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]

could the different addresses be the reason this doesn't work on linux?

[mntyfrsh]

You're misunderstanding how hardware is presented to the operating system kernel. The reason why the PCI addresses are different on the Proxmox host versus the VM is because Proxmox is presenting a virtual motherboard to the VM. Thus the PCI addressing for devices does not have a 1:1 mapping from the reported addressing perspective.

Also, the error you're having with Ubuntu (linux) might be related to the AMD vendor reset bug. If you don't know what it is here's some background info:

https://github.com/gnif/vendor-reset

gnif/vendor-reset#21

https://github.com/inga-lovinde/RadeonResetBugFix (this doesn't work for linux btw, it's Windows only)

[Snuupy]

I've rebooted my host and it still wasn't working. I thought the vendor reset bug is only when you bring down the VM and bring it back up again?

[mntyfrsh]

Guess your problem isn't related to the vendor reset bug. Perhaps you need to install the firmware-adm-graphics to get it working in linux as well? See this link:

#45

[Snuupy]

that might be it. let me reinstall and give it another try. will update this post when I do. thanks!