Re: [Xen-devel] DESIGN: CPUID part 3

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 09/06/17 13:24, Anshul Makkar wrote:
> On 08/06/2017 14:12, Andrew Cooper wrote:
>> Presented herewith is the a plan for the final part of CPUID work, which
>> primarily covers better Xen/Toolstack interaction for configuring the
>> guests
>> CPUID policy.
>>
>> A PDF version of this document is available from:
>>
>> http://xenbits.xen.org/people/andrewcoop/cpuid-part-3.pdf
>>
>> There are a number of still-open questions, which I would appreaciate
>> views
>> on.
>>
>> ~Andrew
>>
>>
>> # Proposal
>>
>> First and foremost, split the current **max\_policy** notion into
>> separate
>> **max** and **default** policies.  This allows for the provision of
>> features
>> which are unused by default, but may be opted in to, both at the
>> hypervisor
>> level and the toolstack level.
>>
>> At the hypervisor level, **max** constitutes all the features Xen can
>> use on
>> the current hardware, while **default** is the subset thereof which are
>> supported features, the features which the user has explicitly opted
>> in to,
>> and excluding any features the user has explicitly opted out of.
>>
>> A new `cpuid=` command line option shall be introduced, whose
>> internals are
>> generated automatically from the featureset ABI.  This means that all
>> features
>> added to `include/public/arch-x86/cpufeatureset.h` automatically gain
>> command
>> line control.  (RFC: The same top level option can probably be used for
>> non-feature CPUID data control, although I can't currently think of
>> any cases
>> where this would be used Also find a sensible way to express
>> 'available but
>> not to be used by Xen', as per the current `smep` and `smap` options.)
>>
>>
>> At the guest level, **max** constitutes all the features which can be
>> offered
>> to each type of guest on this hardware.  Derived from Xen's **default**
>> policy, it includes the supported features and explicitly opted in to
>> features, which are appropriate for the guest.
>>
>> The guests **default** policy is then derived from its **max**, and
>> includes
>> the supported features which are considered migration safe.  (RFC: This
>> distinction is rather fuzzy, but for example it wouldn't include
>> things like
>> ITSC by default, as that is likely to go wrong unless special care is
>> taken.)
>>
> Just from other perspective, what happens to the features which have
> been explicilty selected and are not migration safe ? Do, we consider
> them in guest's default policy.

Explicitly selected where?

Explicit selection at the Xen level is for using experimental/preview
features, while explicit selection at the toolstack level is for both
experimental/preview features, and using features which require more
care wrt migration.

>
>> All global policies (Xen and guest, max and default) shall be made
>> available
>> to the toolstack, in a manner similar to the existing
> Instead of all, do you see any harm if we expose only the default
> policies of Xen and Guest to toolstack.

The entire point of this work is to provide the toolstack with enough
information to work correctly.  Hiding the max policies is not an
option, as it prevents the toolstack from being able to work out whether
it can offer non-default features or not.

>> _XEN\_SYSCTL\_get\_cpu\_featureset_ mechanism.  This allows decisions
>> to be
>> taken which include all CPUID data, not just the feature bitmaps.
>>
>> New _XEN\_DOMCTL\_{get,set}\_cpuid\_policy_ hypercalls will be
>> introduced,
>> which allows the toolstack to query and set the cpuid policy for a
>> specific
>> domain.  It shall supersede _XEN\_DOMCTL\_set\_cpuid_, shall fail if
>> Xen is
>> unhappy with any aspect of the policy during auditing.
>>
>> When a domain is initially created, the appropriate guests
>> **default** policy
>> is duplicated for use.  When auditing, Xen shall audit the toolstacks
>> requested policy against the guests **max** policy.  This allows
>> experimental
>> features or non-migration-safe features to be opted in to, without those
>> features being imposed upon all guests automatically.
>
>>
>> The `disable_migrate` field shall be dropped.  The concept of
>> migrateability
>> is not boolean; it is a large spectrum, all of which needs to be
>> managed by
>> the toolstack.
> Can't this large spectrum result in a bool which can then be used for
> disable_migrate. Sorry, I can't see any value add in removing
> disable_migrate.

A spectrum is by definition not a single boolean.  What is unclear about
my argument here that disable_migrate is unfit for purpose?

~Andrew

>  The simple case is picking the common subset of features
>> between the source and destination.  This becomes more complicated
>> e.g. if the
>> guest uses LBR/LER, at which point the toolstack needs to consider
>> hardware
>> with the same LBR/LER format in addition to just the plain features.
>>
>> `disable_migrate` is currently only used to expose ITSC to guests,
>> but there
>> are cases where is perfectly safe to migrate such a guest, if the
>> destination
>> host has the same TSC frequency or hardware TSC scaling support.
>>
>> Finally, `disable_migrate` doesn't (and cannot reasonably) be used to
>> inhibit
>> state gather operations, as this interferes with debugging and
>> monitoring
>> tasks.
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel