When you configure devices for passthru on ESXi the changes you make are stored in /etc/vmware/esx.conf. This makes it possible to edit your passthru configuration with vifs.pl (supported option) or at the console (unsupported). In the below 2 examples I have a screenshot of the VMDirectPath configuration screen, as well as the output of esx.conf and lspci.
Sample 1
In this example I’ve enabled an audio controller, USB PCI-E card, and Adaptec card for passthru. The Nvidia graphics cards is also enabled as it is on Bus 3 (as is the Adaptec card) of the server. In the output of esx.conf you can see that only the audio controller is specifically set to passthru.
For the other devices it is PCI bridge devices that are enabled and thus all devices on that bus are enabled for passthru. When a VMDirectpath VM is started, ESXi must be able to reset the PCI device and when the devices are on a shared bus all devices end up getting a reset.
If the device is capable of being individually reset, it may be possible to edit /etc/vmware/passthru.map to allow this. In the case of the PCI-E USB card, it has multiple PCI functions thus shows as 3 separate entries. Only one of the functions can be used for VMDirectPath at a time.
esx.conf
/device/000:03.2/owner = "vmkernel"
/device/000:25.0/vmkname = "vmnic0"
/device/000:26.0/owner = "vmkernel"
/device/000:26.1/owner = "vmkernel"
/device/000:26.2/owner = "vmkernel"
/device/000:26.7/owner = "vmkernel"
/device/000:27.0/owner = "passthru"
/device/000:29.0/owner = "vmkernel"
/device/000:29.1/owner = "vmkernel"
/device/000:29.2/owner = "vmkernel"
/device/000:29.7/owner = "vmkernel"
/device/000:30.0/owner = "passthru"
/device/000:31.2/owner = "vmkernel"
/device/000:31.2/vmkname = "vmhba0"
/device/000:31.5/owner = "vmkernel"
/device/000:31.5/vmkname = "vmhba1"
/device/001:00.0/owner = "passthru"
/device/002:00.0/owner = "vmkernel"
/device/002:00.1/owner = "vmkernel"
/device/002:00.2/owner = "vmkernel"
lspci output
00:00.00 Bridge: Intel Corporation 82Q35 Express DRAM Controller
00:02.00 Display controller: Intel Corporation 82Q35 Express Integrated Graphics Controller
00:03.00 Communication controller: Intel Corporation 82Q35 Express MEI Controller
00:03.02 Mass storage controller: Intel Corporation 82Q35 Express PT IDER Controller
00:03.03 Communication controller: Intel Corporation 82Q35 Express Serial KT Controller
00:25.00 Network controller: Intel Corporation 82566DM-2 Gigabit Network Connection [vmnic0]
00:26.00 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4
00:26.01 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5
00:26.02 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6
00:26.07 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2
00:27.00 Multimedia controller: Intel Corporation 82801I (ICH9 Family) HD Audio Controller
00:28.00 Bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1
00:29.00 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1
00:29.01 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2
00:29.02 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3
00:29.07 Serial bus controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1
00:30.00 Bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge
00:31.00 Bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface Controller
00:31.02 Mass storage controller: Intel Corporation 4 port SATA IDE Controller (ICH9) [vmhba0]
00:31.03 Serial bus controller: Intel Corporation 82801I (ICH9 Family) SMBus Controller
00:31.05 Mass storage controller: Intel Corporation 2 port SATA IDE Controller (ICH9) [vmhba1]
00:31.06 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem
01:00.00 Bridge: Texas Instruments XIO2000(A)/XIO2200(A) PCI Express-to-PCI Bridge
02:00.00 Serial bus controller: NEC Corporation USB
02:00.01 Serial bus controller: NEC Corporation USB
02:00.02 Serial bus controller: NEC Corporation USB 2.0
03:00.00 Display controller: nVidia Corporation NV34 [GeForce FX 5200]
03:01.00 Mass storage controller: Adaptec AHA-2940AU Single
Sample 2
In the below example all the USB devices in the system enabled for passthru which created an issue as ESXi was booting from USB.
esx.conf when the system was running
advUserOptions/options[0003]/name = "CIMWatchdogInterval"
/advUserOptions/options[0003]/type = "int"
/devcie/008:01.0/owner = "vmkernel"
/device/000:26.0/owner = "vmkernel"
/device/000:26.1/owner = "vmkernel"
/device/000:26.2/owner = "vmkernel"
/device/000:26.7/owner = "vmkernel"
/device/000:27.0/owner = "passthru"
/device/000:29.0/owner = "vmkernel"
/device/000:29.1/owner = "vmkernel"
/device/000:29.2/owner = "vmkernel"
/device/000:29.7/owner = "vmkernel"
/device/000:30.0/owner = "passthru"
/device/000:31.2/owner = "vmkernel"
/device/000:31.2/vmkname = "vmhba0"
/device/000:31.5/owner = "vmkernel"
/device/000:31.5/vmkname = "vmhba1"
/device/006:00.0/vmkname = "vmnic0"
/device/007:00.0/vmkname = "vmnic1"
/device/008:03.0/vmkname = "vmnic2"
/device/008:03.1/vmkname = "vmnic3"
/net/pnic/child[0000]/mac = "00:30:48:db:68:88"
/net/pnic/child[0000]/name = "vmnic0"
lspci output for passthru devices
00:27.00 Multimedia controller: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:30.00 Bridge: Intel Corporation 82801BA/CA/DB/EB PCI Bridge
esx.conf with the system powered down
The file was taken from /bootbank/local.tgz and the entries highlighted in yellow were assigned to “passthru” before being changed to “vmkernel” as shown in the image.