Re: [Xen-devel] Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)

[Suravee Suthikulanit <suravee.suthikulpanit@xxxxxxx>]

On 8/6/2013 1:55 AM, Jan Beulich wrote:
> > > > On 06.08.13 at 04:02, Suravee Suthikulanit <suravee.suthikulpanit@xxxxxxx>
> wrote:
> > On 8/5/2013 7:58 AM, Jan Beulich wrote:
> > > > > > On 22.07.13 at 14:55, Stefan Bader <stefan.bader@xxxxxxxxxxxxx> wrote:
> > > > While testing Xen 4.3 on my AMD testbox I noticed the following output from
> > > > the
> > > > hypervisor (this has no visible effect on at least simple operation):
> > > >
> > > > (XEN) setup 0000:00:18.0 for d0 failed (-19)
> > > > (XEN) setup 0000:00:18.1 for d0 failed (-19)
> > > > (XEN) setup 0000:00:18.2 for d0 failed (-19)
> > > > (XEN) setup 0000:00:18.3 for d0 failed (-19)
> > > > (XEN) setup 0000:00:18.4 for d0 failed (-19)
> > > > (XEN) setup 0000:00:19.0 for d0 failed (-19)
> > > > (XEN) setup 0000:00:19.1 for d0 failed (-19)
> > > > (XEN) setup 0000:00:19.2 for d0 failed (-19)
> > > > (XEN) setup 0000:00:19.3 for d0 failed (-19)
> > > > (XEN) setup 0000:00:19.4 for d0 failed (-19)
> > > >
> > > > The PCI devices related to the output are all PCI host bridges:
> > > >
> > > > 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor HyperTransport Configuration
> > > > 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Address Map
> > > > 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor DRAM Controller
> > > > 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Miscellaneous Control
> > > > 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Link Control
> > > > 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor HyperTransport Configuration
> > > > 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Address Map
> > > > 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor DRAM Controller
> > > > 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Miscellaneous Control
> > > > 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
> > > >    Family 10h Processor Link Control
> > > >
> > > > This all seems to be related to PCI passthrough setup started from
> > > > xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
> > > > xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
> > > >
> > > > While the Intel code skips over bridge type entries, the AMD code has no
> > > > such
> > > > exception and will fail the handler with -ENODEV when find_iommu_for_device
> > > > fails. But I am not sure one can just compare implementations here.
> > > >
> > > > Would someone have more insight to decide whether skipping host bridges in
> > > > amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
> > > > caused by this but it does not look good and did not happen before.
> > > Fundamentally this looks like a BIOS bug, and hence doing as you
> > > suggest unconditionally is not an option imo.
> > Actually, I don't think this is a BIOS bug.  I have looked through the IVRS
> > table
> > on the system that I have and it looks correct to me in the sense that it
> > does not list
> > all these processor host bridge devices for IOMMU.  As Stefan mentioned, the
> > current
> > dom0 PCI setup code scans through PCI bus does not have filter for the host
> > bridges which
> > do not need IOMMU. However, these error message should not be harmful.
> > Also, in debug mode,
> > it clearly state that "No iommu for device 0000:00:18.0".
> So are you saying that you also see those messages in debug mode?
The default (stock unstable) is showing "(XEN) setup 0000:00:18.0 for d0 failed 
(-19)".
With the "amd-iommu-debug" boot options, it is also showing "(XEN) AMD-Vi: No iommu 
for device 0000:00:18.0".

> If so, i.e. if they are expected to always appear, we should indeed
> try to avoid issuing them, as they may cause unnecessary worries
> (as is the case for Stefan here).
Agree
> > > Adding code to skip host bridges (under the assumption that they won't
> > initiate
> > > transactions that might require translation by the IOMMU) may be
> > > an option if enabled only via command line option.
> > Currently, pdev_type() in driver/passthrough/pci.c only match the
> > PCI_CLASS_BRIDGE_PCI (0x0604)
> > and not the Host/PCI bridge (0x600) which I see what the devices 0:18.x,
> > 0x19.x are reporting in
> > the PCI class code structure. This might need to change?
> We may want to change this, but this needs to be done carefully
> to not cause regressions on the VT-d side (to which host bridges
> currently are the same as ordinary PCI devices/PCIe endpoints).
Which class code are showing on Intel system for host bridges?

Suravee
> Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel