Re: [Xen-devel] RESEND [Xen-unstable][Qemu-xen] HVM Guest reading of Expansion ROM from passthroughed PCI device returns data from emulated VGA rom

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Mon, Dec 02, 2013 at 08:55:57PM +0100, Sander Eikelenboom wrote:
> 
> Monday, December 2, 2013, 8:41:31 PM, you wrote:
> 
> > On Tue, Sep 17, 2013 at 04:33:39PM +0200, Sander Eikelenboom wrote:
> >> *RESEND* due to exceeding the mailinglists limit for attachment size.
> >> 
> >> Hi,
> >> 
> >> I'm trying to get secondary vga-passthrough on a HVM guest to work with a 
> >> AMD HD6570 and the native kernel radeon driver and kernel modesetting.
> >> So the guest still gets the emulated stdvga or cirrus device(used in my 
> >> case here) as primary/boot vga adapter.
> >> 
> >> - When i don't passthrough the radeon card, the linux native radeon driver 
> >> loads fine.
> >> - When i do passtrough the device to a HVM with the same kernel:
> >>   The driver in the guest tries to read the pci expansion rom from the 
> >> passthroughed device to get the vbios.
> >>   The driver reports a successful read, but fails because it can't find 
> >> the right string at the right offset.
> >> 
> >> So I inspected the rom by using sysfs with:
> >> echo 1 > /sys/bus/pci/devices/<BDF>/rom
> >> cat /sys/bus/pci/devices/<BDF>/rom
> >> 
> >> - When i use this in dom0 (so without passthrough) i the contents of the 
> >> ROM are valid (as expected since the driver loads fine)
> >> - When i use this in the Guest (passthrouhed), the contents of the ROM i 
> >> get are not from the passedthrough adapter, but from the emulated cirrus 
> >> card.
> >>   (it's the same as the 
> >> "tools/firmware/vgabios/VGABIOS-lgpl-latest.cirrus.bin")
> 
> > I hadn't tried that, but I do get the same error:
> 
> > [    4.143445] [drm] radeon defaulting to kernel modesetting.
> > [    4.143452] [drm] radeon kernel modesetting enabled.
> > [    4.143525] checking generic (f0000000 160000) vs hw (110000000 10000000)
> > [    4.143976] xen: --> pirq=24 -> irq=40 (gsi=40)
> > [    4.147760] [drm] initializing kernel modesetting (RV770 0x1002:0x9440 
> > 0x174B:0xE850).
> > [    4.148204] [drm] register mmio base: 0xF3040000
> > [    4.148209] [drm] register mmio size: 65536
> > [    4.152035] radeon 0000:00:06.0: >Expecting atombios for R600 GPU
> > [    4.152040] radeon 0000:00:06.0: >Fatal error during GPU init 
> > [    4.152044] [drm] radeon: finishing device.
> > [    4.152047] [TTM] Memory type 2 has not been initialized
> > [    4.168163] radeon 0000:00:06.0: >no bo for sa manager
> > [    4.174380] radeon: probe of 0000:00:06.0 failed with error -22
> 
> > This is with the latest Xen version. No idea yet what is broken.
> 
> Nope i haven't been able to solve it so far.
> Last weekend i tried vga passthrough with KVM and virtio and that worked 
> without much hassle.
> (although i cheated a bit and used the "rom_file" option to let qemu load the 
> vga bios instead of loading it directly from the card)

Aha :-)
> The problem is many parts seems to be involved (hvmloader -> seabios -> qemu 
> -> xen parts of qemu .. and xen as the hypervisor)
> 
> Qemu sets up the rombar, but there seems to be no backing from Xen, i also 
> don't understand al the special casing around the rombar,
> (when you leave out the exception of loading it from while, what is so 
> different from a normal IOMEM region apart from that it should be readonly ?)

If my memory serves me right - the only difference is that there needs
to be a mechanism to "upload" the firmware in hvmloader. That way when
QEMU starts it has the firmware blobls in memory and can execute them.

But that actually is not needed for PCI passthrough devices as . well
,you _pass_ them in. So the ROM should be read from the device. And I
recall somebody doing it with LSI SAS and they found a bug in the BIOS.
Also somebody did it with QLogic FibreChannel card. Would have to dig
up the email archives to find out.
> The virtio driver seems to be somewhat ahead, it also seems handle quirks for 
> some cards already.

virtio or virt-manager (and in extension the libs that do all of the
nasty bits of shuffling bits around, enabling/disabling extensions,
etc?)

> Haven't had much time lately to experiment, but the test with KVM at least 
> showed it should be possible with my card.

/me nods.
> 
> --
> Sander
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel