Re: [Xen-devel] [RFC] xen/pvh: detect PVH after kexec

[Vitaly Kuznetsov <vkuznets@xxxxxxxxxx>]

Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx> writes:

> On 03/21/2017 10:44 AM, Vitaly Kuznetsov wrote:
>> Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> writes:
>>
>>> Roger Pau Monne <roger.pau@xxxxxxxxxx> writes:
>>>
>>>> On Tue, Mar 21, 2017 at 10:21:52AM +0100, Vitaly Kuznetsov wrote:
>>>>> Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx> writes:
>>>>>
>>>>>> On 03/20/2017 02:20 PM, Vitaly Kuznetsov wrote:
>>>>>>> PVH guests after kexec boot like normal HVM guests and we're not 
>>>>>>> entering
>>>>>>> xen_prepare_pvh()
>>>>>> Is it not? Aren't we going via xen_hvm_shutdown() and then
>>>>>> SHUTDOWN_soft_reset which would restart at the same entry point as
>>>>>> regular boot?
>>>>> No, we're not doing regular boot: from outside of the guest we don't
>>>>> really know where the new kernel is placed (as guest does it on its
>>>>> own). We do soft reset to clean things up and then guest jumps to the
>>>>> new kernel starting point by itself.
>>>>>
>>>>> We could (in theory, didn't try) make it jump to the PVH starting point
>>>>> but we'll have to at least prepare the right boot params for
>>>>> init_pvh_bootparams and this looks like additional
>>>>> complication. PVHVM-style startup suits us well but we still need to be
>>>>> PVH-aware.
>>>> We are going to have the same issue when booting PVH with OVMF, Linux will 
>>>> be
>>>> started at the native UEFI entry point, and we will need some way to detect
>>>> that we are running in PVH mode.
>>>>
>>>> What issues do you see when using the HVM boot path for kexec?
>>> The immediate issue I ran into was ballooning driver over-allocating
>>> with XENMEM_populate_physmap:
>>>
>>> (XEN) Dom15 callback via changed to Direct Vector 0xf3
>>> (XEN) d15v0 Over-allocation for domain 15: 262401 > 262400
>>> (XEN) memory.c:225:d15v0 Could not allocate order=0 extent: id=15 
>>> memflags=0 (175 of 512)
>>> (XEN) d15v0 Over-allocation for domain 15: 262401 > 262400
>>> (XEN) memory.c:225:d15v0 Could not allocate order=0 extent: id=15 
>>> memflags=0 (0 of 512)
>>> (XEN) d15v0 Over-allocation for domain 15: 262401 > 262400
>>> ...
>>>
>>> I didn't investigate why it happens, setting xen_pvh=1 helped. Not sure
>>> if it's related, but I see the following code in __gnttab_init():
>>>
>>>     /* Delay grant-table initialization in the PV on HVM case */
>>>     if (xen_hvm_domain() && !xen_pvh_domain())
>>>             return 0;
>>>
>>> and gnttab_init() is later called in platform_pci_probe().
>> Two more things:
>>
>> There is xen_pvh_gnttab_setup() initcall which does 
>>
>>      if (!xen_pvh_domain())
>>                 return -ENODEV;
>>
>> and this is probably the source of over-allocation.
>>
>> xen_has_pv_devices() has the following:
>>
>>         if (xen_pv_domain() || xen_pvh_domain())
>>                 return true;
>>
>> which also has to be patched for PVH-after-kexec. So it seems we either
>> have to remove all this stuff somehow or make PVH-ness detectable...
>>
>
> Can we read xenstore's dm-version? Or is it too early to get access there?
>

Seems to be too late: xenbus_init() is a postcore_initcall. Moreover, it
seems that 'dm-version' lives under 'libxl' in xenstore, not sure if
it's readable from the domain.

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel