[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] RE: pci pass-through failure on xen 3.3.0

Hi Masroor,
There are many important bug fixes in the latest 3.3 tree. Maybe it's better to try the latest 3.3 code. For instance, you may need a small changeset 18381:24083a2dba7f to fix a format typo here. :-)
 

For the co-assignment, if a device(including multi-function device) hasn't a proper standard FLR methods, we have to use the SecondaryBusReset as a FLR method, so we require the co-assignment.

You can refer to http://xenbits.xensource.com/xen-3.3-testing.hg?rev/e61978c24d84 for details.

 
Thanks,

-- Dexuan

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
Sent: 2008年10月10日 10:01
To: 'Masroor Vettuparambil'; 'xen-devel@xxxxxxxxxxxxxxxxxxx'
Subject: [Xen-devel] RE: pci pass-through failure on xen 3.3.0

Hi Masroor,

Thanks for your testing! Glad to see an incorrect assumption is fixed here.

 

For the non-page aligned BAR issue, it's expected:

Your NIC has a non-page aligned BAR:

  Memory at ff5f7800 (64-bit, prefetchable) [disabled] [size=2K]

Currently Xen disallows this kind of assignment due to some known issues...

 

What's lucky is:

Recently Yuji Shimada has posted a patch to resolve the case.

Can you try the latest Dom0(changeset 693:2b5cc22ab406)?

 

 Thanks,

-- Dexuan

From: Masroor Vettuparambil [mailto:Masroor.Vettuparambil@xxxxxxxxxxxx]
Sent: 2008年10月10日 9:43
To: Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: pci pass-through failure on xen 3.3.0

Hi Dexuan,

 

Thanks for your patch. It looks like the below problem fixed.

But now I ended up with “non-page aligned bar address” error.

Xend log and lspci config space are given below,

 

By the way, I could see in the script that all the pci devices behind the bridge should be co-assigned to the same guest.

Does it mean that we could not export each device function on a multi-function card to separate guests?

 

Xend log is,

 

[2008-10-09 18:07:16 7867] ERROR (xmlrpclib2:178) Internal error handling xend.domain.device_configure

Traceback (most recent call last):

  File "//usr/lib64/python/xen/util/xmlrpclib2.py", line 131, in _marshaled_dispatch

    response = self._dispatch(method, params)

  File "/usr/lib64/python2.5/SimpleXMLRPCServer.py", line 415, in _dispatch

    return func(*params)

  File "<string>", line 1, in <lambda>

  File "//usr/lib64/python/xen/xend/server/XMLRPCServer.py", line 55, in dispatch

    return getattr(info, fn)(*args)

  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 821, in device_configure

    return self.pci_device_configure(dev_sxp)

  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 722, in pci_device_configure

    self.device_create(dev_sxp)

  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 637, in device_create

    self._createDevice(dev_type, dev_config_dict)

  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 1869, in _createDevice

    return self.getDeviceController(deviceClass).createDevice(devConfig)

  File "//usr/lib64/python/xen/xend/server/DevController.py", line 94, in createDevice

    self.setupDevice(config)

  File "//usr/lib64/python/xen/xend/server/pciif.py", line 431, in setupDevice

    self.setupOneDevice(domain, bus, slot, func)

  File "//usr/lib64/python/xen/xend/server/pciif.py", line 289, in setupOneDevice

    raise VmError("pci: %: non-page-aligned MMIO BAR found." % dev.name)

ValueError: unsupported format character \047:\047 (0x3a) at index 6

 

Config space dump of the NIC is,

linux-242:# lspci -d 17d5: -v -xxx

08:01.0 Ethernet controller: S2io Inc. Xframe II 10Gbps Ethernet (rev 02)

        Subsystem: Hewlett-Packard Company Unknown device 1337

        Flags: 66MHz, medium devsel, IRQ 9

        Memory at ff5f8000 (64-bit, prefetchable) [disabled] [size=32K]

        Memory at ff400000 (64-bit, prefetchable) [disabled] [size=1M]

        Memory at ff5f7800 (64-bit, prefetchable) [disabled] [size=2K]

        Expansion ROM at fea00000 [disabled] [size=1M]

        Capabilities: [40] Message Signalled Interrupts: Mask+ 64bit+ Queue=0/5 Enable-

        Capabilities: [60] PCI-X non-bridge device

        Capabilities: [80] Vital Product Data

        Capabilities: [90] MSI-X: Enable- Mask- TabSize=64

        Capabilities: [a0] Power Management version 3

00: d5 17 32 58 00 00 30 02 02 00 00 02 10 40 00 00

10: 0c 80 5f ff 00 00 00 00 0c 00 40 ff 00 00 00 00

20: 0c 78 5f ff 00 00 00 00 00 00 00 00 3c 10 37 13

30: 00 00 a0 fe 40 00 00 00 00 00 00 00 09 01 ff 01

40: 05 60 8a 01 00 00 00 00 00 00 00 00 00 00 00 00

50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

60: 07 80 73 10 08 08 e3 5f 00 00 00 00 00 00 00 00

70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

80: 03 90 fc 80 00 00 00 78 00 00 00 00 00 00 00 00

90: 11 a0 3f 00 04 00 00 00 04 04 00 00 00 00 00 00

a0: 01 00 23 7e 00 20 00 dc 00 00 00 00 00 00 00 00

b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

 

Thanks,

Masroor

 

From: Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx]
Sent: Wednesday, October 08, 2008 2:31 AM
To: Masroor Vettuparambil; 'xen-devel@xxxxxxxxxxxxxxxxxxx'
Subject: RE: pci pass-through failure on xen 3.3.0

 

Hi Masroor,

BTW, you can simply apply the patch directly: login the target host, cd /usr/lib64/python/xen, patch -p4 < fix_find_parent.patch; xend restart.

 

Thanks,

-- Dexuan

 

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
Sent: 2008年10月8日 17:16
To: 'Masroor Vettuparambil'; 'xen-devel@xxxxxxxxxxxxxxxxxxx'
Subject: [Xen-devel] RE: pci pass-through failure on xen 3.3.0

Hi Masroor,

Looks there is an incorrect assumption.

Could you please help to test the attached patch? If it doesn't work, can you attached your output of "lspci -tv" and "lspci -xxx -vvv"?

 

Thanks!

 

-- Dexuan

 

 

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Masroor Vettuparambil
Sent: 2008年10月8日 8:33
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] pci pass-through failure on xen 3.3.0

Hi all,

 

I downloaded the xen 3.3.0 and now testing the pci pass-through with the NIC.

Using late binding, the pci slot  0000:08:01.0 is assigned to pciback.

But pci related xm commands are not working.

 

For eg:

linux-242:/home/xen-3.3.0 # xm pci-list-assignable-devices

Unexpected error: <type 'exceptions.OSError'>

 

Please report to xen-devel@xxxxxxxxxxxxxxxxxxx

Traceback (most recent call last):

  File "/usr/sbin/xm", line 10, in <module>

    main.main(sys.argv)

  File "//usr/lib64/python/xen/xm/main.py", line 2883, in main

    _, rc = _run_cmd(cmd, cmd_name, args)

  File "//usr/lib64/python/xen/xm/main.py", line 2907, in _run_cmd

    return True, cmd(args)

  File "//usr/lib64/python/xen/xm/main.py", line 2202, in xm_pci_list_assignable_devices

    devs_list = check_FLR_capability(dev_list)

  File "/usr/lib64/python2.5/xen/util/pci.py", line 288, in check_FLR_capability

    coassigned_pci_list = dev.find_coassigned_devices(True)

  File "/usr/lib64/python2.5/xen/util/pci.py", line 461, in find_coassigned_devices

    dev = self.find_the_uppermost_pci_bridge()

  File "/usr/lib64/python2.5/xen/util/pci.py", line 424, in find_the_uppermost_pci_bridge

    dev_parent = PciDevice(dom, b, d, f)

  File "/usr/lib64/python2.5/xen/util/pci.py", line 382, in __init__

    self.detect_dev_info()

  File "/usr/lib64/python2.5/xen/util/pci.py", line 604, in detect_dev_info

    class_dev = self.pci_conf_read16(PCI_CLASS_DEVICE)

  File "/usr/lib64/python2.5/xen/util/pci.py", line 567, in pci_conf_read16

    fd = os.open(self.cfg_space_path, os.O_RDONLY)

OSError: [Errno 2] No such file or directory: '/sys/bus/pci/devices/0000:06:00.0/config'

 

I checked the above python scripts.

The function find_parent() in file util/pci.py, tries to find the parent device from the file link ../../../devices/pci0000:06/0000:06:01.0/0000:08:01.0

When it sees pci0000:06, it tries to read the config space of 0000:06:00.0 which doesn’t exist. This causes the above error.

 

Lspci tree is,

 

linux-242:/home/xen-3.3.0 # lspci -t

-+-[0000:06]-+-01.0-[0000:08]----01.0

 |           +-01.1

 |           +-02.0-[0000:07]--

 |           \-02.1

 \-[0000:00]-+-00.0

             +-01.0

             +-01.1

             +-06.0

             +-07.0

             +-08.0

             +-09.0-[0000:05]----07.0

             +-0b.0-[0000:04]----00.0

             +-0c.0-[0000:03]----00.0

             +-0d.0-[0000:02]--

             +-0e.0-[0000:01]--

             +-18.0

             +-18.1

             +-18.2

             +-18.3

             +-19.0

             +-19.1

             +-19.2

             \-19.3

 

Thanks,

Masroor

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.