Re: [Xen-users] pvgrub2(-like?) booting methods for PVHv2 guests

[Juergen Gross <jgross@xxxxxxxx>]

On 07/02/18 14:14, Hans van Kranenburg wrote:
> On 02/07/2018 09:37 AM, Juergen Gross wrote:
>> On 06/02/18 19:34, Hans van Kranenburg wrote:
>>> On 02/06/2018 06:02 PM, Juergen Gross wrote:
>>>> On 06/02/18 17:53, Hans van Kranenburg wrote:
>>>>> On 02/06/2018 09:35 AM, Juergen Gross wrote:
>>>>>> On 05/02/18 20:49, Hans van Kranenburg wrote:
>>>>>>> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>>>>>>>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>>>>>>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>>>>>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>>>>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>>>>>>>> Hi Hans,
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> And now I get console output and things happen. Only it can't 
>>>>>>>>>>>>> find the disk.
>>>>>>>>>>>>
>>>>>>>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>>>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>>>>>>>
>>>>>>>>>>>>     
>>>>>>>>>>>> <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>>>>>>>
>>>>>>>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>>>>>>>> okay outside grub with direct kernel boot.
>>>>>>>>>>>>
>>>>>>>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>>>>>>>
>>>>>>>>>>>>     
>>>>>>>>>>>> <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>>>>>>>
>>>>>>>>>>>> But I think you have this don't you?
>>>>>>>>>>
>>>>>>>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>>>>>>>
>>>>>>>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>>>>>>>
>>>>>>>>>> dmesg output is pretty different when I boot directly with the kernel
>>>>>>>>>> and initrd copied on the dom0.
>>>>>>>>>>
>>>>>>>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>>>>>>>> boots with all vcpus network and disk.
>>>>>>>>>>
>>>>>>>>>> With grub in between, this at least does look suspicious:
>>>>>>>>>>
>>>>>>>>>> [    0.032110] PCI: System does not support PCI
>>>>>>>>>
>>>>>>>>> Eh, that's in both, stay awake Hans.
>>>>>>>>>
>>>>>>>>>> And yes, there are also no successful netfront lines.
>>>>>>>>>>
>>>>>>>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>>>>>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>>>>>>>
>>>>>>>>> Or the diff between them, which shows what changes when inserting grub
>>>>>>>>> in between:
>>>>>>>>>
>>>>>>>>> http://paste.debian.net/plainh/52b2d618
>>>>>>>>>
>>>>>>>>> I must admit I don't know too much (yet) about all those changed 
>>>>>>>>> lines,
>>>>>>>>> but this next is also a very interesting change?...
>>>>>>>>>
>>>>>>>>> -Booting paravirtualized kernel on Xen PVH
>>>>>>>>> +Booting paravirtualized kernel on Xen HVM
>>>>>>>>
>>>>>>>> Aah, yes, this should be the reason for the problems.
>>>>>>>>
>>>>>>>> I addressed the ACPI problem first. What is missing now is to set PVH
>>>>>>>> mode when booting via grub.
>>>>>>>>
>>>>>>>> So please stay tuned...
>>>>>>>
>>>>>>> For my info... Is this something like flipping a bit somewhere, or will
>>>>>>> it involve a whole lot more complicated wizardry?
>>>>>>>
>>>>>>> Real question is of course if there's a PoC solution to apply that I can
>>>>>>> use now? ;-] I'm throwing custom build kernels at my test environment,
>>>>>>> and if I could save the time (and mistakes) of manually editing the
>>>>>>> guest files and instead hit the pvgrub2+PVHv2 path many times to test...
>>>>>>
>>>>>> So I looked into this briefly and discovered that my memory really needs
>>>>>> the backup I have on my hard disk: you want commit
>>>>>> 418492ba40b2c2bbdaf1a169aac5b1673bde8189 which was for 4.15.
>>>>>
>>>>> Aha. Well, I can as well better jump to 4.15 now, since picking that
>>>>> patch on 4.14.17 shows I need more other patches it depends on, related
>>>>> to struct x86_legacy_features and x86_hyper_init reorganization.
>>>>>
>>>>> So, I just built a 4.15.1 kernel (using gcc 7.2.0), which definitely has
>>>>> this one in it:
>>>>>   418492ba40b2 x86/virt/xen: Use guest_late_init to detect Xen PVH guest
>>>>>
>>>>> I can boot via pvgrub2 (the Xen that's running now is current
>>>>> stable-4.10 plus the RSDP for PVH guest near 4GB patch).
>>>>>
>>>>> But, it still tells me...
>>>>>   Booting paravirtualized kernel on Xen HVM
>>>>>
>>>>> ...and then hangs somewhere halfway.
>>>>>
>>>>> Full boot log here: http://paste.debian.net/plainh/ae3ea14b
>>>>
>>>> You didn't add the RSDP detection patch, right? In the log I see:
>>>>
>>>> ACPI BIOS Error (bug): A valid RSDP was not found (20170831/tbxfroot-244)
>>>
>>> Yes, I didn't have those. Argh...
>>>
>>> However, after putting the 'xen: re-enable booting as Xen PVH guest' v2
>>> patches on top, same happens.
>>
>> Sure. Those don't take RSDP from boot parameters set by grub2.
>>
>> You need to apply the 4 patch series from:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/boot
>>
>> Commit-Ids are:
>> 2f74cbf947f45fa082dda8eac1a1f1299a372f49
>> 0c89cf36424f7c1177de8a5712514d7cc2eb369f
>> 88750a6c33f813b815516990f01fb5ee488c477e
>> 930ba49b2ce7b09a5eddc21385fd944ba6b4e829
> 
> Ah, D'OH. The patch sets currently bite each other. And, for some reason
> I messed up and was wrongly thinking I already had those in 4.15, since
> which is entirely not the case. No idea why I thought that.
> 
> So, since I'm testing pvgrub2, I threw the 'xen: re-enable booting as
> Xen PVH guest' out again, and applied those 4 again on 4.15.1.
> 
> Now I at least have this...
> [    0.000000] Booting paravirtualized kernel on Xen PVH
> ...and no more 'A valid RSDP was not found'...
> 
> ...but quite some complaining going on about ACPI, like 'BIOS bug: APIC
> version mismatch' and 'ACPI BIOS Warning (bug): Incorrect checksum in
> table' however, and per cpu there's a 10 second boot delay (didn't see
> that one before yet), and I have only 1 vcpu in the end etc...
> 
> Full output:
> http://paste.debian.net/plainh/262cfbcf

e820: BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000000000000000-0x00000000fbffffff] usable
BIOS-e820: [mem 0x00000000fc000000-0x00000000fc000fff] ACPI data
BIOS-e820: [mem 0x00000000fc001000-0x00000000fc008fff] usable
BIOS-e820: [mem 0x00000000fc009000-0x00000000fc009fff] ACPI data
BIOS-e820: [mem 0x00000000fc00a000-0x00000000fedfffff] usable

and

ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000000FC009000 000024 (v02 Xen   )
ACPI: XSDT 0x00000000FC007FB0 000034 (v01 Xen HVM 00000000 HVML 00000000)
ACPI: FACP 0x00000000FC007D70 00010C (v05 Xen HVM 00000000 HVML 00000000)
ACPI: DSDT 0x00000000FC001050 000003 (v164 �mn FE942AD6 ?    FC001510)
ACPI:      0x00000000FC001010 000001 (v08 <-   6DB08FA4      62757267)
ACPI:      0x00000000FC001010 000001 (v08 <-   6DB08FA4      62757267)
ACPI: APIC 0x00000000FC007E80 00007C (v02 Xen HVM 00000000 HVML 00000000)

Seems as if some ACPI data areas are missing in the E820 map.

Can you try configuring the guest with e.g. 3GB memory instead of 4GB?


Juergen

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users