Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10

[Jan Beulich <jbeulich@xxxxxxxx>]

On 28.09.2019 05:07, Roman Shaposhnik wrote:
> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
>>
>> On 26.09.2019 00:31, Roman Shaposhnik wrote:
>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
>>> those and here's where I am.
>>>
>>> First of all, it is definitely related to CPU bring up. Adding
>>> cpuidle=0 to xen command line made Xen boot.
>>>
>>> Then, a good friend of mine (who you may know from ancient Xen days
>>> ;-)) suggested that this could be related to this:
>>>      https://wiki.xenproject.org/wiki/Xen_power_management
>>> so I went to the BIOS settings and quite to my surprise all of them
>>> were grayed out (not tweakable).
>>>
>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
>>> disabled that.
>>>
>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>>
>> Interesting, but unfortunately this particular log is of no real use
>> for investigation of the issue (other than knowing the CPU model). I
>> also notice it's a 4.12.0 log, when your original report was against
>> latest master.
>>
>>> So I guess at this point, you could say that I have a functional
>>> system, but I'm curious whether you guys would be interested to look
>>> into 2xAPIC situation.
>>
>> Of course we do. As a next step I'd suggest reverting the BIOS settings
>> change you did, and instead using the "x2apic=0" Xen command line option.
> 
> Interestingly enough, this doesn't really solve the problem completely.
> Specifying x2apic=0 certainly makes Xen go much further to a point
> where it tries to load Dom0 and then the console VGA screen goes
> blank (this is where that serial debug output would be very useful :-().

Now that's again unexpected. In any event you could try "vga=keep".

>> And then we of course need a complete boot log (as requested earlier) of
>> a problem case.
>>
>> Further I'd suggest moving away from the black-and-white "cpuidle="
>> option, and instead limiting use of deep C states ("max_cstate="). I
>> wouldn't be surprised if this was the issue; we'd then have to first
>> of all go through errata for the part your system is using.
> 
> Yup. max_cstate=1 makes it boot fine. max_cstate=2 though hangs
> the system *exactly* in the same way as specifying x2apic=0
> (which is different from the original problem as I've described above).

"max_cstate=2" is much less of a "deep" C state than I had expected,
but well, so be it then. As to the hang - did you meanwhile figure
whether _any_ number of CPUs above 1 would result in a hang, or
whether instead there's a certain amount of them that would allow
boot to progress fine.

> Can you please elaborate on "we'd then have to first of all go through
> errata for the part your system is using"

Well, it wouldn't be the first time that hardware had issues with C
state handling. Therefore we'd need to (a) be sure you use up-to-date
microcode and (b) there are no errata documented for your CPU model
workarounds for which basically suggest to avoid use of deep C states.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel