Re: [Xen-devel] [PATCH RFC v2] Add SUPPORT.md

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 26/10/17 10:19, Jan Beulich wrote:
>>>> On 25.10.17 at 13:30, <andrew.cooper3@xxxxxxxxxx> wrote:
>> On 25/10/17 11:59, George Dunlap wrote:
>>>>>>>>> +    Limit, x86 HVM: 128
>>>>>>>>> +    Limit, ARM32: 8
>>>>>>>>> +    Limit, ARM64: 128
>>>>>>>>> +
>>>>>>>>> +[XXX Andrew Cooper: Do want to add "Limit-Security" here for some of 
>>>>>>>>> these?]
>>>>>>>> 32 for each.  64 vcpu HVM guests can excerpt enough p2m lock pressure 
>>>>>>>> to
>>>>>>>> trigger a 5 second host watchdog timeout.
>>>>>>> Is that "32 for x86 PV and x86 HVM", or "32 for x86 HVM and ARM64"?  Or
>>>>>>> something else?
>>>>>> The former.  I'm not qualified to comment on any of the ARM limits.
>>>>>>
>>>>>> There are several non-trivial for_each_vcpu() loops in the domain_kill
>>>>>> path which aren't handled by continuations.  ISTR 128 vcpus is enough to
>>>>>> trip a watchdog timeout when freeing pagetables.
>>>>> I don't think 32 is a really practical limit.
>>>> What do you mean by practical here, and what evidence are you basing
>>>> this on?
>>>>
>>>> Amongst other things, there is an ABI boundary in Xen at 32 vcpus, and
>>>> given how often it is broken in Linux, its clear that there isn't
>>>> regular testing happening beyond this limit.
>>> Is that true for dom0 as well?
>> Yes.  The problem is:
>>
>> struct shared_info {
>>     struct vcpu_info vcpu_info[XEN_LEGACY_MAX_VCPUS];
>> ...
>>
>> and while there are ways to make a larger number of vcpus work, it
>> requires additional hypercalls to make alternate arrangements for the
>> vcpus beyond the 32 boundary, and these arrangements appear to be broken
>> more often than not around suspend/resume.
> But I guess the implied part of George's question was: Wouldn't
> we expect Dom0 to be more frequently tested with > 32 vCPU-s,
> as quite likely not everyone has dom0_max_vcpus= in place?

I'm going to make a wild guess and say the intersection of people with
server class hardware and not using dom0_max_vcpus= is very small.

XenServer for example tops out at 16 dom0 vcpus, because performance
(aggregate disk/network throughput) plateaus at that point, and extra
cpu resource is far better spent running the VMs.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel