Re: [Xen-devel] [PATCH v2 1/4] x86/dom0: prevent access to MMCFG areas for PVH Dom0

[Chao Gao <chao.gao@xxxxxxxxx>]

On Tue, Aug 29, 2017 at 08:33:25AM +0100, Roger Pau Monne wrote:
>On Mon, Aug 28, 2017 at 06:18:13AM +0000, Tian, Kevin wrote:
>> > From: Roger Pau Monne [mailto:roger.pau@xxxxxxxxxx]
>> > Sent: Friday, August 25, 2017 9:59 PM
>> > 
>> > On Fri, Aug 25, 2017 at 06:25:36AM -0600, Jan Beulich wrote:
>> > > >>> On 25.08.17 at 14:15, <roger.pau@xxxxxxxxxx> wrote:
>> > > > On Wed, Aug 23, 2017 at 02:16:38AM -0600, Jan Beulich wrote:
>> > > >> >>> On 22.08.17 at 15:54, <roger.pau@xxxxxxxxxx> wrote:
>> > > >> > On Tue, Aug 22, 2017 at 06:26:23AM -0600, Jan Beulich wrote:
>> > > >> >> >>> On 11.08.17 at 18:43, <roger.pau@xxxxxxxxxx> wrote:
>> > > >> >> > --- a/xen/arch/x86/dom0_build.c
>> > > >> >> > +++ b/xen/arch/x86/dom0_build.c
>> > > >> >> > @@ -440,6 +440,10 @@ int __init
>> > dom0_setup_permissions(struct domain *d)
>> > > >> >> >              rc |= rangeset_add_singleton(mmio_ro_ranges, mfn);
>> > > >> >> >      }
>> > > >> >> >
>> > > >> >> > +    /* For PVH prevent access to the MMCFG areas. */
>> > > >> >> > +    if ( dom0_pvh )
>> > > >> >> > +        rc |= pci_mmcfg_set_domain_permissions(d);
>> > > >> >>
>> > > >> >> What about ones reported by Dom0 later on? Which then raises the
>> > > >> >> question whether ...
>> > > >> >
>> > > >> > This should be dealt with in the PHYSDEVOP_pci_mmcfg_reserved
>> > handler.
>> > > >> > But since you propose to do white listing, I guess it doesn't matter
>> > > >> > that much anymore.
>> > > >>
>> > > >> Well, a fundamental question is whether white listing would work in
>> > > >> the first place. I could see room for severe problems e.g. with ACPI
>> > > >> methods wanting to access MMIO that's not described by any PCI
>> > > >> devices' BARs. Typically that would be regions in the chipset which
>> > > >> firmware is responsible for configuring/managing, the addresses of
>> > > >> which can be found/set in custom config space registers.
>> > > >
>> > > > The question would also be what would Xen allow in such white-listing.
>> > > > Obviously you can get to map the same using both white-list and
>> > > > black-listing (see below).
>> > >
>> > > Not really - what you've said there regarding MMCFG regions is
>> > > a clear indication that we should _not_ map reserved regions, i.e.
>> > > it would need to be full white listing with perhaps just the PCI
>> > > device BARs being handled automatically.
>> > 
>> > I've tried just mapping the BARs and that sadly doesn't work, the box
>> > hangs after the IOMMU is enabled:
>> > 
>> > [...]
>> > (XEN) [VT-D]d0:PCI: map 0000:3f:13.5
>> > (XEN) [VT-D]d0:PCI: map 0000:3f:13.6
>> > (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021b000
>> > 
>> > I will park this ATM and leave it for the Intel guys to diagnose.
>> > 
>> > For the reference, the specific box I'm testing ATM has a Xeon(R) CPU
>> > E5-1607 0 @ 3.00GHz and a C600/X79 chipset.
>> > 
>> 
>> +Chao who can help check whether we have such a box at hand.
>> 
>> btw please also give your BIOS version.
>
>It's a Precision T3600 BIOS A14.

Hi, Roger.

I found a Ivy bridge box with E5-2697 v2 and tested with "dom0=pvh", and
the bug didn't occur on this box. The log is below:
XEN) [    7.509588] [VT-D]d0:PCIe: map 0000:ff:1e.2
(XEN) [    7.511047] [VT-D]d0:PCIe: map 0000:ff:1e.3
(XEN) [    7.512463] [VT-D]d0:PCIe: map 0000:ff:1e.4
(XEN) [    7.513927] [VT-D]d0:PCIe: map 0000:ff:1e.5
(XEN) [    7.515342] [VT-D]d0:PCIe: map 0000:ff:1e.6
(XEN) [    7.516808] [VT-D]d0:PCIe: map 0000:ff:1e.7
(XEN) [    7.519449] [VT-D]iommu_enable_translation: iommu->reg =
ffff82c00021b000
(XEN) [    7.522295] [VT-D]iommu_enable_translation: iommu->reg =
ffff82c00021d000
(XEN) [    8.675096] OS: linux version: 2.6 loader: generic bitness:
64-bit
(XEN) [    8.726763] 
(XEN) [    8.730171] ****************************************
(XEN) [    8.737491] Panic on CPU 0:
(XEN) [    8.742376] Building a PVHv2 Dom0 is not yet supported.
(XEN) [    8.750148] ****************************************
(XEN) [    8.757457] 
(XEN) [    8.760877] Reboot in five seconds...
(XEN) [   13.769050] Resetting with ACPI MEMORY or I/O RESET_REG

I agree with you that there may be some bugs in firmware and VT-d.
I did trials on a haswell box with iommu_inclusive_mapping=false. I did
see DMA traslation fault. The address to be translated is reserved in
e820 but isn't included in RMRR. Even that, the box booted up
successfully.

But if the bug exists in pvh dom0, it also exists in pv dom0. Could you
try to boot with pv dom0 and set iommu_inclusive_mapping=false?
Theoretically, the system would halt exactly like what you met in
pvh dom0. Is that right? or I miss some difference between pvh dom0 and
pv dom0.

Thanks
Chao

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel