Re: [Xen-devel] q35 support in Xen

[Alexey G <x1917x@xxxxxxxxx>]

Hi,

> >> On Mon, Jun 26, 2017 at 01:55:56PM -0400, Jason Dickens wrote:  
> >>> I would like to inquire about q35 support in Xen? As far as I have been
> >>> able to tell, this has not been done? In the Xen version that I've been
> >>> working with (4.4), libxl_dm overrides any "-machine" argument I try to
> >>> pass to QEMU with "-machine xenfv". (it appears this still existing in
> >>> the last version)
> >>>
> >>> In my case, I need q35 support because certain OVMF functionality requires
> >>> the q35 architecture.  
> > By curiosity, which functionality of OVMF ?  
> I'm trying to get SMM functionality in OVMF, this only works with q35. I 
> have since been informed that Xen may not support SMM mode in the 
> firmware anyway and that adding it could be a significant effort.
> >>> Can someone help with:
> >>>
> >>> 1. Is there a newer version of Xen that does support q35 emulation?  
> > No.
> >  
> >>> 2. Has there been a determination of what has to change for q35? e.g.,
> >>> just ACPI?  
> > There is also some simple change in QEMU, about interrupt I think, and
> > we need to teach hvmloader to recognize the new platform and do some
> > initialisation.  
> I think the interrupt thing is support for SMI handlers in the firmware?
> >>> 3. Are there plans to support this?  
> > I don't think there is. I did work on it in the past but it was not a
> > priority. But patchs are always welcomed.  
> I would like to work towards full support of SMM in OVMF while on Xen 
> and may well do some development in that area. I would certainly welcome 
> a discussion and collaboration.


Well, SMI emulation for Q35 in QEMU might disappoint you (looks like QEMU
lacks emulation of many Q35 features at the moment), but regarding your first
question -- Q35 support can be added to Xen in fact. Q35 provides some features
which might be useful, don't know why its support haven't been added yet - i440
is a pretty much legacy platform.

Anthony Perard did a great job providing patches which add a partial Q35
support. I tried extending his patches to include missing features for Q35 in
hvmloader, libacpi and QEMU and so far the Xen+Q35 experience is quite positive
-- all basic functionality like MP, storage (via AHCI), networking, Xen
Platform dev, powering off/rebooting VMs and so on works OK. In fact, disk I/O
speed in a guest with AHCI seems to be a bit better than disk I/O via IDE, both
without a PV xen block device... although this needs some real benchmarking
and PV block device is still better of course.

The only troublesome issue with Xen+Q35 was adding passthrough support for
Windows 7+ guest OSes. Starting with Windows 7, windows handles PCIe devices
very differently when it encounters a PCIe-capable system. Without taking this
into account, there will be a "Code 10" error for any passed through PCIe
device in guest Windows 7+ no matter the fact that Windows successfully booted
actually _using_ this device, ex. as a primary VGA card even with VBE features
working.

The good solution for this problem would be to rework PCI-related code in
libxl/hvmloader/QEMU a bit to provide a more realistic view of a PCI bus in
guest VMs but this will require some effort...
Luckily, there is a simple workaround which allows to bypass this problem and
to use PCIe passthrough on Q35 normally until PCI handling code will be
upgraded.

Another (not big though) issue is that Q35 emulation in QEMU is far from
complete. So we have to use some facilities in an old i440-way, as a temporary
solution - ex. to perform ACPI soft-off like we are on a i440 system. From a
guest OS perspective everything is as usual - it performs soft-off via ACPI _S5
etc, but IIRC its actual emulation differs from a real Q35 chipset and is done
by the same piece of code as for i440.

Another minor issue with Q35 support in Xen is that Xen is limited to max 4
PIRQs in multiple places, while Q35 have support of 8 PIRQs. I workarounded
this by describing only 4 usable IRQ link entries in DSDT -- like we're
on a real system which has only some of 8 available PIRQs physically connected
to the chipset (should be typical for many desktop PCs anyway).

The more serious issue found on Q35 was a bug in xen-mapcache with corrupting
mapcache entries due to intensive async DMA I/O via AHCI controller -- there
was a xen-list thread with a more detailed description a couple of months ago.
The issue has a very low reproducibility rate and that's the main problem - how
to catch the bug which appears ex. once in a week. I wrote a test driver for
guest windows which performs sort of fuzzing the DMA emulation code in QEMU
-- but using available IDE (ATA and ATAPI) devices for DMA I/O instead of AHCI.
Unfortunately, the result was not the one I expected - the test driver simply
crashes QEMU due to ASSERT in IDE emulation code, not due to the xen-mapcache
bug, so the test code must be adapted to AHCI.

AFAIR Stefano Stabellini made a fix for a similar xen-mapcache issue recently
that might be possibly fixed the DMA issue as well, but this still needs to be
verified on AHCI.

Xen+Q35+OVMF might require some additional work... I remember some issue with
dependency on VM memory size with OVMF, but can't say for sure if it was
Q35 only. I did all my experiments with Q35 using SeaBIOS.

Anyway, basic Q35 support can be added to Xen as an optional feature - Anthony
suggested adding a 'machine=q35'config parameter which will allow enabling Q35
support on demand while leaving the old machine by default. At this stage Q35
support needs a lot of testing and collecting further bugs.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel