Re: [Xen-devel] Regression, host crash with 4.5rc1

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 27.02.15 at 18:50, <len.brown@xxxxxxxxx> wrote:
> If this issue were to happen on Linux/bare-metal, this is how I'd debug it.
> Hopefully some of this will translate to Xen in one way or another.

Sadly not really - the kernel plays only a minor role (forwarding ACPI
data to the hypervisor) in C-state handling under Xen.

> dmesg | grep idle
> will tell us what idle driver is running (on Dom0 kernel)
> and if it is intel_idle, it will also tell us the supported sub-states 
> (CPUID.MWAIT.EDX value)

Yeah, we call the driver mwait-idle in the hypervisor, and the log
would be accssible via "xl dmesg", but yes, that information is
available there too.

>> > (XEN)     C1:   type[C1] latency[003] usage[12219860] method[  FFH]
>> > duration[1190961948551]
>> > (XEN)     C2:   type[C1] latency[010] usage[10205554] method[  FFH]
>> > duration[2015393965907]
>> > (XEN)     C3:   type[C2] latency[020] usage[50926286] method[  FFH]
>> > duration[30527997858148]
> 
> I'm hopeful that this information comes from the hardware's BIOS
> and not some hypervisor tricking out Dom0 with a fake BIOS, yes?

In the case of mwait-idle (intel_idle on Linux) it would be built-in
knowledge of the driver. For acpi-cpuidle it would come from
actual firmware, not anything fake/virtual.

> Next, hopefully the attached turbostat utility can be invoked on Dom0
> and it can read the MSRs on at least 1 processor via the /dev/cpu interface.

Yes, that would be possible, provided it's not important what specific
CPU it gets executed on.

> It may tell us just the same thing I think we learned here:
> 
>> > (XEN) PC2[0] PC3[8589642315848] PC6[0] PC7[0]
>> > (XEN) CC3[28794734145697] CC6[0] CC7[0]
> 
> which I'm assuming are a dump of the MSR residency counters.
> If yes, it appears to be that this platform is not invoking c6 and pc6 at 
> all,
> and that the deepest state being used is actually cc3 and pc3.
> I don't know if that is because you've booted the kernel with max_cstate=N
> of some kind, or if this is default.

Sadly I haven't been able to tell which original mail the quotes
above are from, and since I had Steve experiment with disabling
the deepest C-state permitted to be used, it may well be that
this output was from one of those experiments. Remember, we
already know that with use of C6 alone disabled things work for
him (Steve - please correct me if I'm misremembering).

> Guessing...
> If no surprises in the debug stuff requested above, and
> If the XEN debug stuff above is with c6 explicitly disabled...
> Note that here are two kinds of c6 -- CC6 (core) and PC6 (package).
> If this box supports both, the next thing to try will be to keep CC6
> enabled, but to just disable PC6.  This is done via an MSR that turbostat
> dumps out (MSR_NHM_SNB_PKG_CST_CFG_CTL) via the wrmsr(8) utility.

I don't think the wrmsr tool can be used (unmodified) to reliably do
this on all CPUs in the system - we'd likely have to cook up a patch
to the hypervisor instead, or I'd have to hand my patch to msr-tools
to Steve so he could use the tool under Xen (albeit that would also
require him to use one of our forward ported kernels, as the
upstream one doesn't have a pCPU sysfs interface yet afaik).

> Though if that MSR is locked by the BIOS, then BIOS SETUP option
> may be the only way to disable the package C-state limit without
> also disabling the associated core C-state.

Steve, could you check whether any such option exists (it's been
a while, so apologies if we had asked already)?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel