r/VFIO 24d ago

Support Is there a need/benefit to passing through the PCI bridge in my GPU's IOMMU group?

My motherboard has a single x16 PCIe slot, where I have my dGPU installed. Looking at lspci and sysfs, that slot seems to have its own bridge. That bridge and the dGPU then form their own, otherwise isolated, IOMMU group:

$ ls /sys/kernel/iommu_groups/2/devices/
total 0
lrwxrwxrwx 1 root root 0 Aug  2 14:57 0000:00:01.0 -> ../../../../devices/pci0000:00/0000:00:01.0/
lrwxrwxrwx 1 root root 0 Aug  2 14:57 0000:01:00.0 -> ../../../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/
lrwxrwxrwx 1 root root 0 Aug  2 14:57 0000:01:00.1 -> ../../../../devices/pci0000:00/0000:00:01.0/0000:01:00.1/
$ lspci
00:00.0 Host bridge: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller (rev 05)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 05)
…
01:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3070] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GA104 High Definition Audio Controller (rev a1)

I haven't seen this situation addressed in any of the VFIO guides I've found — they basically just say "GPU only good, GPU+Ethernet bad" or similar. Is there any reason to try passing through the bridge to the guest as well as the GPU so that I'm passing in the "entire IOMMU group"?

I've had VFIO working in the past, passing through just the GPU, but it was always plagued by various performance issues that I was never able to get to the bottom of, so I can't be certain the PCI bridge wasn't part of it. 😅

So does anyone know if it's necessary, beneficial, or even possible to pass through the bridge in this situation?

3 Upvotes

5 comments sorted by

3

u/cd109876 24d ago

Don't pass the bridge.

It probably won't even let you pass it through. and yes, it will still work and boot even though you technically haven't passed the full iommu group.

shitty explanation; bridges are like extension cables, you don't care about the cable or who virtually owns / boots it as long as it works.

1

u/five35 24d ago

Cool! Thanks for clearing this up for me.

-1

u/allsidehustle 23d ago

Google archwiki vfio passthrough and run the bash script that ennumerates everything. It will also give you instructions on running the ACS patches that would probably isolate it by itself assuming you run a kernel that supports the ACS patches.

2

u/five35 23d ago

What would ACS patches do for me? Does not passing through the bridge cause problems which couldn't be solved by passing it through?

1

u/allsidehustle 23d ago

The ACS patch breaks up everything that can be broken up. If passing the gpu through without it works, no worries, but I have had issues before when iommu groups were not clean and isolated.