Re: [Xen-devel] [DRAFT RFC] PVHv2 interaction with physical devices

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Nov 10, 2016 at 09:37:19AM -0700, Jan Beulich wrote:
> >>> On 10.11.16 at 11:39, <roger.pau@xxxxxxxxxx> wrote:
> > On Wed, Nov 09, 2016 at 01:45:17PM -0500, Konrad Rzeszutek Wilk wrote:
> >> On Wed, Nov 09, 2016 at 04:59:12PM +0100, Roger Pau Monné wrote:
> >> > PCI memory BARs
> >> > ---------------
> >> > 
> >> > PCI devices discovered by Xen will have it's BARs scanned in order to 
> >> > detect
> >> > memory BARs, and those will be identity mapped to Dom0. Since BARs can be
> >> > freely moved by the Dom0 OS by writing to the appropriate PCI config 
> >> > space
> >> > register, Xen must trap those accesses and unmap the previous region and
> >> > map the new one as set by Dom0.
> >> 
> >> You can make that simpler - we have hypercalls to "notify" in Linux
> >> when a device is changing. Those can provide that information as well.
> >> (This is what PV dom0 does).
> >> 
> >> Also you are missing one important part - the MMCFG. That is required
> >> for Xen to be able to poke at the PCI configuration spaces (above the 256).
> >> And you can only get the MMCFG if the ACPI DSDT has been parsed.
> > 
> > Hm, I guess I'm missing something, but at least on my hardware Xen seems to 
> > be able to parse the MCFG ACPI table before Dom0 does anything with the 
> > DSDT:
> > 
> > (XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
> > (XEN) PCI: MCFG area at f8000000 reserved in E820
> 
> This is the crucial line: To guard against broken firmware, we - just
> like Linux - require that the area be reserved in at least one of E820
> or ACPI resources. We can check E820 ourselves, but we need
> Dom0's AML parser for the other mechanism.
> 
> >> So if you do the PCI bus scanning _before_ booting PVH dom0, you may
> >> need to update your view of PCI devices after the MMCFG locations
> >> have been provided to you.
> > 
> > I'm not opposed to keep the PHYSDEVOP_pci_mmcfg_reserved, but I still have 
> > to see hardware where this is actually needed. Also, AFAICT, FreeBSD at 
> > least is only able to detect MMCFG regions present in the MCFG ACPI table:
> > 
> > http://fxr.watson.org/fxr/source/dev/acpica/acpi.c?im=excerp#L1861 
> 
> Iirc the spec mandates only segment 0 to be represented in the
> static table. Other segments may (and likely will) only have their
> data available in AML.

I don't mind leaving the PHYSDEVOP_pci_mmcfg_reserved hypercall, but it _must_ 
be issued before Dom0 tries to actually access the MCFG area, or else it won't 
be mapped into Dom0 p2m.

> >> > XXX: is it possible to have more than 256 GSIs?
> >> 
> >> Yeah. If you have enough of the IOAPICs you can have more than 256. But
> >> I don't think any OS has taken that into account as the GSI value are
> >> always uint8_t.
> > 
> > Right, so AFAICT providing a single IO APIC with enough pins should be fine.
> 
> No, let's not even start with such an approach. Having seen (not really
> huge) systems with well beyond 100 GSIs, I don't think it makes sense
> to try to (temporarily) ease our lives slightly by introducing an
> implementation limit here.

OK, the only limit I see here is that ACPI GSI numbers are encoded in a double 
word in _PRT objects, so there can theoretically be systems out there with up 
to 
65536 GSIs. I very much doubt there's any systems out there with more than 256 
GSIs, but better be safe than sorry.

I assume that temporary limiting PVHv2 Dom0 support to systems with 1 IO APIC 
only is not going to be accepted, right?

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel