Xen project Mailing List

[Xen-devel] [PATCH v11 04/23] x86: refactor psr: L3 CAT: implement main data structures, CPU init and free flows.

To construct an extendible framework, we need analyze PSR features and abstract the common things and feature specific things. Then, encapsulate them into different data structures. By analyzing PSR features, we can get below map. +------+------+------+ --------->| Dom0 | Dom1 | ... | | +------+------+------+ | | |Dom ID | cos_id of domain | V | +-----------------------------------------------------------------------------+ User --------->| PSR | Socket ID | +--------------+---------------+---------------+ | | | Socket0 Info | Socket 1 Info | ... | | | +--------------+---------------+---------------+ | | | cos_id=0 cos_id=1 ... | | | +-----------------------+-----------------------+-----------+ | | |->Ref : | ref 0 | ref 1 | ... | | | | +-----------------------+-----------------------+-----------+ | | | +-----------------------+-----------------------+-----------+ | | |->L3 CAT: | cos 0 | cos 1 | ... | | | | +-----------------------+-----------------------+-----------+ | | | +-----------------------+-----------------------+-----------+ | | |->L2 CAT: | cos 0 | cos 1 | ... | | | | +-----------------------+-----------------------+-----------+ | | | +-----------+-----------+-----------+-----------+-----------+ | | |->CDP : | cos0 code | cos0 data | cos1 code | cos1 data | ... | | | +-----------+-----------+-----------+-----------+-----------+ | +-----------------------------------------------------------------------------+ So, we need define a socket info data structure, 'struct psr_socket_info' to manage information per socket. It contains a reference count array according to COS ID and a feature array to manage all features enabled. Every entry of the reference count array is used to record how many domains are using the COS registers according to the COS ID. For example, L3 CAT and L2 CAT are enabled, Dom1 uses COS_ID=1 registers of both features to save CBM values, like below. +-------+-------+-------+-----+ | COS 0 | COS 1 | COS 2 | ... | +-------+-------+-------+-----+ L3 CAT | 0x7ff | 0x1ff | ... | ... | +-------+-------+-------+-----+ L2 CAT | 0xff | 0xff | ... | ... | +-------+-------+-------+-----+ If Dom2 has same CBM values, it can reuse these registers which COS_ID=1. That means, both Dom1 and Dom2 use same COS registers(ID=1) to keep same L3/L2 values. So, the value of ref[1] is 2 which means 2 domains are using COS_ID 1. To manage a feature, we need define a feature node data structure, 'struct feat_node', to manage feature's specific HW info, and an array of all COS registers values of this feature. To manage feature properties, we need define a feature property data structure, 'struct feat_props', to manage common properties (callback functions - all feature's specific behaviors are encapsulated into these callback functions, and generic values - e.g. the cos_max), the feature independent values. CDP is a special feature which uses two entries of the array for one COS ID. So, the number of CDP COS registers is the half of L3 CAT. E.g. L3 CAT has 16 COS registers, then CDP has 8 COS registers if it is enabled. CDP uses the COS registers array as below. +-----------+-----------+-----------+-----------+-----------+ CDP cos_reg_val[] index: | 0 | 1 | 2 | 3 | ... | +-----------+-----------+-----------+-----------+-----------+ value: | cos0 code | cos0 data | cos1 code | cos1 data | ... | +-----------+-----------+-----------+-----------+-----------+ For more details, please refer SDM and patches to implement 'get value' and 'set value'. This patch also implements the CPU init and free flow including L3 CAT initialization and some resources free. It includes below flows: 1. presmp init: - parse command line parameter. - allocate socket info for every socket. - allocate feature resource. - initialize socket info, get feature info and add feature into feature array per cpuid result. - free resources allocated if error happens. - register cpu notifier to handle cpu events. 2. cpu notifier: - handle cpu online events, if initialization work has been done before, do nothing. - handle cpu offline events, if it is the last cpu offline, free some socket resources. Signed-off-by: Yi Sun <yi.y.sun@xxxxxxxxxxxxxxx> --- v11: - handle 'feat_init'. - merge main data structures implementation into CPU init/free patch, including commit messages and change history. (suggested by Jan Beulich) - remove MSR restore action which is unnecessary. (suggested by Jan Beulich) - move 'type[]' declaration into this patch. (suggested by Jan Beulich) - modify comment. (suggested by Jan Beulich) v10: - remove initialization for 'PSR_SOCKET_L3_CAT'. (suggested by Jan Beulich) - rename 'feat_ops' to 'feat_props'. (suggested by Jan Beulich) - move 'cbm_len' to 'feat_props' because it is feature independent so far. (suggested by Jan Beulich) - move 'cos_max' to 'feat_props' because it is feature independent. (suggested by Jan Beulich) - move 'cos_num' to 'feat_props' because it is feature independent. (suggested by Jan Beulich) - remove union 'info' and struct 'psr_cat_hw_info'. - remove 'get_cos_max' from 'feat_props'. (suggested by Jan Beulich) - remove 'feat_mask' from 'psr_socket_info' because we can use 'features[]' to check if any feature is initialized. (suggested by Jan Beulich) - move 'ref_lock' above 'cos_ref'. (suggested by Jan Beulich) - adjust comments and commit message according to above changes. - remove 'asm/x86_emulate.h' inclusion as it has been indirectly included. (suggested by Jan Beulich) - remove 'CAT_COS_NUM' as it is only used once. (suggested by Jan Beulich) - remove 'feat_mask'. (suggested by Jan Beulich) - changes about 'feat_props'. (suggested by Jan Beulich) - remove 'get_cos_max' hook declaration. (suggested by Jan Beulich) - modify 'cat_default_val' implementation. (suggested by Jan Beulich) - modify 'psr_alloc_feat_enabled' implementation to make it simple. (suggested by Jan Beulich) - rename 'free_feature' to 'free_socket_resources' because it is executed when socket is offline. It needs free resources related to the socket. (suggested by Jan Beulich) - define 'feat_init_done' to iterate feature array to check if any feature has been initialized. (suggested by Jan Beulich) - input 'struct cpuid_leaf' pointer into 'cat_init_feature' to avoid memory copy. (suggested by Jan Beulich) - modify 'cat_init_feature' to use switch and things related to above changes. (suggested by Jan Beulich) - add an indentation for label. (suggested by Jan Beulich) v9: - replace feature list to a feature pointer array. (suggested by Roger Pau) - add 'PSR_SOCKET_MAX_FEAT' in 'enum psr_feat_type' to know features account. (suggested by Roger Pau) - move 'feat_ops' declaration into 'feat_node' structure. (suggested by Roger Pau) - directly use uninon for feature HW info and move its declaration into 'feat_node' structure. (suggested by Roger Pau) - remove 'enum psr_feat_type feature' declared in 'feat_ops' because it is not useful after using feature pointer array. (suggested by Roger Pau) - rename 'l3_cat_info' to 'cat_info' to be used by all CAT/CDP features. - remove 'nr_feat' which is only for a record. (suggested by Jan Beulich) - add 'cos_num' to record how many COS registers are used by a feature in one time access. (suggested by Jan Beulich) - replace 'uint64_t' to 'uint32_t' for cbm value because SDM specifies the max 32 bits for it. (suggested by Jan Beulich) - add commit message to explain the flows. - handle cpu offline and online again case to read MSRs registers values back and save them into cos array to make user can get real data. - create a new patch about moving 'cpuid_count_leaf'. (suggested by Wei Liu) - modify comment to explain why not free some resource in 'free_feature'. (suggested by Wei Liu) - implement 'psr_alloc_feat_enabled' to check if allocation feature is enabled in cmdline and some initialization work done. (suggested by Wei Liu) - implement 'cat_default_val' to set default value for CAT features. (suggested by Wei Liu) - replace feature list handling to feature array handling. (suggested by Roger Pau) - implement a common 'cat_init_feature' to replace L3 CAT/L2 CAT specific init functions. (suggested by Roger Pau) - modify comments for global feature node. (suggested by Jan Beulich) - remove unnecessary comments. (suggested by Jan Beulich) - remove unnecessary 'else'. (suggested by Jan Beulich) - remove 'nr_feat'. (suggested by Jan Beulich) - modify patch title to indicate 'L3 CAT'. (suggested by Jan Beulich) - check global flag with boot cpu operations. (suggested by Jan Beulich) - remove 'cpu_init_work' and move codes into 'psr_cpu_init'. (suggested by Jan Beulich) - remove 'cpu_fini_work' and move codes into 'psr_cpu_fini'. (suggested by Jan Beulich) - assign value for 'cos_num'. (suggested by Jan Beulich) - change about 'uint64_t' to 'uint32_t'. (suggested by Jan Beulich) v8: - fix format issue. (suggested by Konrad Rzeszutek Wilk) - add comments to explain why we care about cpumask_empty when the last cpu on socket is offline. (suggested by Konrad Rzeszutek Wilk) v7: - sort inclusion files position. (suggested by Wei Liu) - initialize structure objects for avoiding surprise. (suggested by Konrad Rzeszutek Wilk) - fix typo. (suggested by Konrad Rzeszutek Wilk) - fix a logical mistake when handling the last cpu offline event. (suggested by Konrad Rzeszutek Wilk) v6: - make commit message be clearer. (suggested by Konrad Rzeszutek Wilk) - fix wordings. (suggested by Konrad Rzeszutek Wilk) - add comments to explain relationship between 'feat_mask' and 'enum psr_feat_type'. (suggested by Konrad Rzeszutek Wilk) - use 'struct cpuid_leaf' introduced in Andrew's patch. (suggested by Konrad Rzeszutek Wilk) - add comments about cpu_add_remove_lock. (suggested by Konrad Rzeszutek Wilk) - change 'clear_bit' to '__clear_bit'. (suggested by Konrad Rzeszutek Wilk) - add 'ASSERT' check when setting 'feat_mask'. (suggested by Konrad Rzeszutek Wilk) - adjust 'printk' position to avoid odd spacing. (suggested by Konrad Rzeszutek Wilk) - add comment to explain usage of 'feat_l3_cat'. (suggested by Konrad Rzeszutek Wilk) - fix wording. (suggested by Konrad Rzeszutek Wilk) - move 'cpuid_count_leaf' helper function to 'asm-x86/processor.h'. It cannot be moved to 'cpuid.h' which causes compilation error because of header file loop reference. (suggested by Andrew Cooper) v5: - remove section number. (suggested by Jan Beulich) - remove double blank. (suggested by Jan Beulich) - add comment to explain the reason to define 'feat_l3_cat'. (suggested by Jan Beulich) - use 'list_for_each_entry_safe'. (suggested by Jan Beulich) - remove codes to free 'feat_l3_cat' in 'free_feature' to avoid the need for an allocation the next time a CPU comes online. (suggested by Jan Beulich) - define 'struct cpuid_leaf_regs' to encapsulate eax~edx. (suggested by Jan Beulich) - print feature info on a socket only when 'opt_cpu_info' is true. (suggested by Jan Beulich) - declare global variable 'l3_cat_ops' to 'static const'. (suggested by Jan Beulich) - use 'current_cpu_data'. (suggested by Jan Beulich) - rename 'feat_tmp' to 'feat'. (suggested by Jan Beulich) - clear PQE feature bit when the maximum CPUID level is too low. (suggested by Jan Beulich) - directly call 'l3_cat_init_feature'. No need to make it a callback function. (suggested by Jan Beulich) - remove local variable 'info'. (suggested by Jan Beulich) - move 'INIT_LIST_HEAD' into 'cpu_init_work' to be together with spin_lock_init(). (suggested by Jan Beulich) - remove 'cpu_prepare_work' and move its content into 'psr_cpu_prepare'. (suggested by Jan Beulich) v4: - create this patch because of removing all old CAT/CDP codes to make implementation be more easily understood. (suggested by Jan Beulich) --- xen/arch/x86/psr.c | 277 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 271 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 96a8589..b73856e 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -13,16 +13,111 @@ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for * more details. */ -#include <xen/init.h> #include <xen/cpu.h> #include <xen/err.h> +#include <xen/init.h> #include <xen/sched.h> #include <asm/psr.h> +/* + * Terminology: + * - CAT Cache Allocation Technology + * - CBM Capacity BitMasks + * - CDP Code and Data Prioritization + * - CMT Cache Monitoring Technology + * - COS/CLOS Class of Service. Also mean COS registers. + * - COS_MAX Max number of COS for the feature (minus 1) + * - MSRs Machine Specific Registers + * - PSR Intel Platform Shared Resource + */ + #define PSR_CMT (1<<0) #define PSR_CAT (1<<1) #define PSR_CDP (1<<2) +#define CAT_CBM_LEN_MASK 0x1f +#define CAT_COS_MAX_MASK 0xffff + +/* + * Per SDM chapter 'Cache Allocation Technology: Cache Mask Configuration', + * the MSRs ranging from 0C90H through 0D0FH (inclusive), enables support for + * up to 128 L3 CAT Classes of Service. The COS_ID=[0,127]. + * + * The MSRs ranging from 0D10H through 0D4FH (inclusive), enables support for + * up to 64 L2 CAT COS. The COS_ID=[0,63]. + * + * So, the maximum COS register count of one feature is 128. + */ +#define MAX_COS_REG_CNT 128 + +/* + * Every PSR feature uses some COS registers for each COS ID, e.g. CDP uses 2 + * COS registers (DATA and CODE) for one COS ID, but CAT uses 1 COS register. + * We use below macro as the max number of COS registers used by all features. + * So far, it is 2 which means CDP's COS registers number. + */ +#define PSR_MAX_COS_NUM 2 + +enum psr_feat_type { + PSR_SOCKET_L3_CAT, + PSR_SOCKET_FEAT_NUM, +}; + +/* + * This structure represents one feature. + * cos_max - The max COS registers number got through CPUID. + * cbm_len - The length of CBM got through CPUID. + * cos_reg_val - Array to store the values of COS registers. One entry stores + * the value of one COS register. + * For L3 CAT and L2 CAT, one entry corresponds to one COS_ID. + * For CDP, two entries correspond to one COS_ID. E.g. + * COS_ID=0 corresponds to cos_reg_val[0] (Data) and + * cos_reg_val[1] (Code). + */ +struct feat_node { + /* cos_max and cbm_len are common values for all features so far. */ + unsigned int cos_max; + unsigned int cbm_len; + uint32_t cos_reg_val[MAX_COS_REG_CNT]; +}; + +/* + * This structure defines feature specific values, e.g. cos_num. + * + * Array 'feat_props' is defined to save every feature's properties. We use + * 'enum psr_feat_type' as index. + */ +static const struct feat_props { + /* + * cos_num - COS registers number that feature uses for one COS ID. + * It is defined in SDM. + */ + unsigned int cos_num; + + /* + * An array to save all 'enum cbm_type' values of the feature. It is + * used with cos_num together to get/write a feature's COS registers + * values one by one. + */ + enum cbm_type type[PSR_MAX_COS_NUM]; +} *feat_props[PSR_SOCKET_FEAT_NUM]; + +/* + * PSR features are managed per socket. Below structure defines the members + * used to manage these features. + * ref_lock - A lock to protect cos_ref. + * features - A feature node array used to manage all features enabled. + * cos_ref - A reference count array to record how many domains are using the + * COS ID. Every entry of cos_ref corresponds to one COS ID. + */ +struct psr_socket_info { + bool feat_init; + spinlock_t ref_lock; + /* Feature array's index is 'enum psr_feat_type' which is same as 'props' */ + struct feat_node *features[PSR_SOCKET_FEAT_NUM]; + unsigned int cos_ref[MAX_COS_REG_CNT]; +}; + struct psr_assoc { uint64_t val; uint64_t cos_mask; @@ -30,11 +125,95 @@ struct psr_assoc { struct psr_cmt *__read_mostly psr_cmt; +static struct psr_socket_info *__read_mostly socket_info; + static unsigned int opt_psr; static unsigned int __initdata opt_rmid_max = 255; +static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT; static uint64_t rmid_mask; static DEFINE_PER_CPU(struct psr_assoc, psr_assoc); +/* + * Declare global feature node for every feature to facilitate the feature + * array creation. It is used to transiently store a spare node. + */ +static struct feat_node *feat_l3_cat; + +/* Common functions */ +#define cat_default_val(len) (0xffffffff >> (32 - (len))) + +/* + * Use this function to check if any allocation feature has been enabled + * in cmdline. + */ +static bool psr_alloc_feat_enabled(void) +{ + return !!socket_info; +} + +static void free_socket_resources(unsigned int socket) +{ + unsigned int i; + struct psr_socket_info *info = socket_info + socket; + + if ( !info ) + return; + + /* + * Free resources of features. The global feature object, e.g. feat_l3_cat, + * may not be freed here if it is not added into array. It is simply being + * kept until the next CPU online attempt. + */ + for ( i = 0; i < PSR_SOCKET_FEAT_NUM; i++ ) + { + xfree(info->features[i]); + info->features[i] = NULL; + } + + info->feat_init = false; +} + +/* CAT common functions implementation. */ +static void cat_init_feature(const struct cpuid_leaf *regs, + struct feat_node *feat, + struct psr_socket_info *info, + enum psr_feat_type type) +{ + /* No valid value so do not enable feature. */ + if ( !regs->a || !regs->d ) + return; + + feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1; + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK); + + switch ( type ) + { + case PSR_SOCKET_L3_CAT: + /* cos=0 is reserved as default cbm(all bits within cbm_len are 1). */ + feat->cos_reg_val[0] = cat_default_val(feat->cbm_len); + + break; + + default: + return; + } + + /* Add this feature into array. */ + info->features[type] = feat; + + if ( !opt_cpu_info ) + return; + + printk(XENLOG_INFO "%s CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n", + ((type == PSR_SOCKET_L3_CAT) ? "L3" : "L2"), + cpu_to_socket(smp_processor_id()), feat->cos_max, feat->cbm_len); +} + +/* L3 CAT props */ +static struct feat_props l3_cat_props = { + .cos_num = 1, +}; + static void __init parse_psr_bool(char *s, char *value, char *feature, unsigned int mask) { @@ -74,6 +253,9 @@ static void __init parse_psr_param(char *s) if ( val_str && !strcmp(s, "rmid_max") ) opt_rmid_max = simple_strtoul(val_str, NULL, 0); + if ( val_str && !strcmp(s, "cos_max") ) + opt_cos_max = simple_strtoul(val_str, NULL, 0); + s = ss + 1; } while ( ss ); } @@ -229,19 +411,98 @@ void psr_domain_free(struct domain *d) psr_free_rmid(d); } -static int psr_cpu_prepare(unsigned int cpu) +static void __init init_psr(void) +{ + if ( opt_cos_max < 1 ) + { + printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n"); + return; + } + + socket_info = xzalloc_array(struct psr_socket_info, nr_sockets); + + if ( !socket_info ) + { + printk(XENLOG_INFO "Failed to alloc socket_info!\n"); + return; + } +} + +static void __init psr_free(void) { + xfree(socket_info); + socket_info = NULL; +} + +static int psr_cpu_prepare(void) +{ + if ( !psr_alloc_feat_enabled() ) + return 0; + + /* Malloc memory for the global feature node here. */ + if ( feat_l3_cat == NULL && + (feat_l3_cat = xzalloc(struct feat_node)) == NULL ) + return -ENOMEM; + return 0; } static void psr_cpu_init(void) { + struct psr_socket_info *info; + unsigned int socket; + unsigned int cpu = smp_processor_id(); + struct feat_node *feat; + struct cpuid_leaf regs; + + if ( !psr_alloc_feat_enabled() || !boot_cpu_has(X86_FEATURE_PQE) ) + goto assoc_init; + + if ( boot_cpu_data.cpuid_level < PSR_CPUID_LEVEL_CAT ) + { + setup_clear_cpu_cap(X86_FEATURE_PQE); + goto assoc_init; + } + + socket = cpu_to_socket(cpu); + info = socket_info + socket; + if ( info->feat_init ) + goto assoc_init; + + spin_lock_init(&info->ref_lock); + + cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, &regs); + if ( regs.b & PSR_RESOURCE_TYPE_L3 ) + { + cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, &regs); + + feat = feat_l3_cat; + feat_l3_cat = NULL; + l3_cat_props.type[0] = PSR_CBM_TYPE_L3; + feat_props[PSR_SOCKET_L3_CAT] = &l3_cat_props; + + cat_init_feature(&regs, feat, info, PSR_SOCKET_L3_CAT); + + info->feat_init = true; + } + + assoc_init: psr_assoc_init(); } static void psr_cpu_fini(unsigned int cpu) { - return; + unsigned int socket = cpu_to_socket(cpu); + + if ( !psr_alloc_feat_enabled() ) + return; + + /* + * We only free when we are the last CPU in the socket. The socket_cpumask + * is cleared prior to this notification code by remove_siblinginfo(). + */ + if ( socket_cpumask[socket] && cpumask_empty(socket_cpumask[socket]) ) + free_socket_resources(socket); } static int cpu_callback( @@ -253,7 +514,7 @@ static int cpu_callback( switch ( action ) { case CPU_UP_PREPARE: - rc = psr_cpu_prepare(cpu); + rc = psr_cpu_prepare(); break; case CPU_STARTING: psr_cpu_init(); @@ -282,10 +543,14 @@ static int __init psr_presmp_init(void) if ( (opt_psr & PSR_CMT) && opt_rmid_max ) init_psr_cmt(opt_rmid_max); - psr_cpu_prepare(0); + if ( opt_psr & PSR_CAT ) + init_psr(); + + if ( psr_cpu_prepare() ) + psr_free(); psr_cpu_init(); - if ( psr_cmt_enabled() ) + if ( psr_cmt_enabled() || psr_alloc_feat_enabled() ) register_cpu_notifier(&cpu_nfb); return 0; -- 1.9.1 _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx https://lists.xen.org/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.