Xen project Mailing List

Eric, could you provide your motherboard information and as Andrew pointed out whatâs f0:1f.0?

From: Andrew Cooper [mailto:andrew.cooper3@xxxxxxxxxx]
Sent: Sunday, November 29, 2015 6:15 AM
To: Eric Shelton; xen-devel; Zhang, Yang Z; Tian, Kevin; Keir Fraser; Jan Beulich
Subject: Re: Skylake: VT-d and other error messages

On 28/11/15 20:46, Eric Shelton wrote:

Looking through the output of 'xl dmesg' on a Skylake system (i5-6600K), I found a number of error messages that I do not encounter on a Haswell-based system. I have tried two motherboards from different manufacturers, with pretty much the same results. Below are some of the unexpected messages:

Not enabling x2APIC (upon firmware request)

...

mwait-idle: does not run on family 6 model 94

...

[VT-D] iommu.c:875: iommu_fault_status: Primary Pending Fault

[VT-D] INTR-REMAP: Request device [0000:f0:1f.0] fault index 0, iommu reg = ffff82c000201000

(on motherboard 1) [VT-D] INTR-REMAP: reason 22 - Present field in the IRTE entry is clear

(on motherboard 2) [VT-D] INTR-REMAP: reason 25 - Blocked a compatibility format interrupt request

This leads to a few questions:

1) Is there some reason x2APIC should not be enabled on Skylake? What consequence, if any, is there not having x2APIC enabled?

In this case, the firmware has set the x2apic opt-out bit in the DMAR table, indicating that Xen should not use x2apic.

You might find an option in your BIOS to undo this; there have been enough errata in the past in this area that I would expect it to be a tweakable.

x2apic is the extension to xapic, which permits more than 255 cpus. So long as you don't have that many, there isn't a specific problem with missing x2apic mode.

2) Should mwait-idle be available on Skylake?

We probably need to resync the mwait driver with Linux. It is whitelisted on known cpu model numbers.

3) What about the IOMMU errors on Skylake - are they a concern?

Yes.

In both cases, PCI device f0:1f.0 is misconfigured or misbehaving.

On motherboard 1, it is delivering an interrupt for which no remapping entry has been set up.

On motherboard 2, it is delivering an compatibility-format interrupt, as opposed to a remapable-format interrupt.

For motherboard 2, Xen should disallow such a configuration. Either interrupt remapping is enabled and all devices should be configured to issue remmapable interrupts, or interrupt remapping is disabled and everything should be configured to issue compatibility-format interrupts.

Either way, diagnosing the problem here starts with identifying what f0:1f.0 is.

~Andrew

Re: [Xen-devel] Skylake: VT-d and other error messages