RHEL and pciback aka hiding pci card from Xen DOM0

Xen PCI Pass thru

I just recently spent more hours than I care to think about getting a NIC in Xen Dom0 hidden and passed to a DomU
 via pciback in RHEL .

Lesson number 1: pciback or pciback.hide is now a module. Putting it on GRUB’s kernel line is no longer sufficien
t.

Lesson number 2: If you NIC module does not load until late in the boot process either use another one or put it
in your initrd. Davicom cards are a good example of this. Use an Intel.

Lesson number 3: Much of the information that you find online about what to put in your /etc/modprobe.config inco
rrect.

So now that we’ve covered all of that here is what you need to do to use pciback in RHEL5 .

Step 1. Put the following and only the following in your /etc/modprobe.conf:

options pciback hide=(01:06.0)

The numbers that you see there are found by running lspci |grep ethernet (or whatever you want to hide such as vg
a, usb, etc).

[root@virtual-host xen]# lspci |grep -i ethernet
01:02.0 Ethernet controller: Davicom Semiconductor, Inc. 21×4x DEC-Tulip compatible 10/100 Ethernet (rev 31)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
[root@virtual-host xen]#

Now you need to create a new initrd image. Before doing so make a backup copy of the one that you are currently using. Then create a new initrd so that the pciback module is loaded early on.

mkinitrd -f –preload=pciback /boot/initrd-$(uname -r).img $(uname -r)

This will place the new initrd into your boot directory and overwrite the old one.

Moving on to the /etc/xen directory we have a little work do here. In the domU config file make two entries like the following:

pci = [ "01:06.0"]
vif = [ '' ]

Now open /etc/xen/xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
#(’0123:4567:89AB:CDEF’)
(’8086:100e’) ##Everything but this entry is already in here for an example.
)

You get the 8086:100e number from running lspci -n:

You get the 8086:100e number from running lspci -n:

[root@virtual-host xen]# lspci -n
00:00.0 0600: 8086:254c (rev 01)
00:00.1 ff00: 8086:2541 (rev 01)
00:1d.0 0c03: 8086:2482 (rev 02)
00:1d.1 0c03: 8086:2484 (rev 02)
00:1e.0 0604: 8086:244e (rev 42)
00:1f.0 0601: 8086:2480 (rev 02)
00:1f.1 0101: 8086:248b (rev 02)
00:1f.3 0c05: 8086:2483 (rev 02)
01:02.0 0200: 1282:9102 (rev 31)
01:04.0 0300: 1002:4752 (rev 27)
01:06.0 0200: 8086:100e (rev 02)
As you can see those number match up with the 01:06.0 number that we used in modprobe.conf and the domU config file.

Now if all has gone well you should see that your domU has direct access to the pci card and that dom0 no longer attempts to use it. This can be confirmed by grep’ing dmesg:

[root@virtual-host xen]# dmesg |grep pciback
pciback 0000:01:06.0: seizing device
pciback 0000:01:06.0: enabling permissive mode configuration space accesses!
pciback 0000:01:06.0: permissive mode is potentially unsafe!
pciback: vpci: 0000:01:06.0: assign to virtual slot 0
pciback: vpci: 0000:01:06.0: assign to virtual slot 0
[root@virtual-host xen]#

There you have the results of several hours of reading and trial and error. Don’t forget that if you install a new xen kernel you will again have to make a new initrd.

One way around having to create a new initrd is to create a file named pciback under /etc/sysconfig/mkinitrd. Put an entry into the pciback file that reads PREMODS=”$PREMODS pciback”. That should automatically install the pciback module when your new kernel creates it’s new mkinitrd.

Hope this helps.