Re: [Xen-devel] [PATCH v13 01/23] docs: create Cache Allocation Technology (CAT) and Code and Data Prioritization (CDP) feature document

[Meng Xu <xumengpanda@xxxxxxxxx>]

[I just add some of my thoughts when I read this document.]

> +## Hardware perspective
> +
> +  CAT/CDP defines a range of MSRs to assign different cache access patterns
> +  which are known as CBMs, each CBM is associated with a COS.
> +
> +  ```
> +  E.g. L2 CAT:
> +                          +----------------------------+----------------+
> +     IA32_PQR_ASSOC       | MSR (per socket)           |    Address     |
> +   +----+---+-------+     +----------------------------+----------------+
> +   |    |COS|       |     | IA32_L2_QOS_MASK_0         |     0xD10      |
> +   +----+---+-------+     +----------------------------+----------------+
> +          └-------------> | ...                        |  ...           |
> +                          +----------------------------+----------------+
> +                          | IA32_L2_QOS_MASK_n         | 0xD10+n (n<64) |
> +                          +----------------------------+----------------+
> +  ```
> +
> +  L3 CAT/CDP uses a range of MSRs from 0xC90 ~ 0xC90+n (n<128).
> +
> +  L2 CAT uses a range of MSRs from 0xD10 ~ 0xD10+n (n<64), following the L3
> +  CAT/CDP MSRs, setting different L2 cache accessing patterns from L3 cache 
> is
> +  supported.
> +
> +  Every MSR stores a CBM value. A capacity bitmask (CBM) provides a hint to 
> the
> +  hardware indicating the cache space a domain should be limited to as well 
> as
> +  providing an indication of overlap and isolation in the CAT-capable cache 
> from
> +  other domains contending for the cache.
> +
> +  Sample cache capacity bitmasks for a bitlength of 8 are shown below. Please
> +  note that all (and only) contiguous '1' combinations are allowed (e.g. 
> FFFFH,
> +  0FF0H, 003CH, etc.).

IIRC, the number of contiguous '1's in CBM should be at least 2 at
least on my machine (Intel(R) Xeon(R) CPU E5-2618L v3).
I'm unsure if this constraint exist for all CAT-capable processors.
For those processors that have such constraint, the system may crash
when the user sets only 1 bit to the CBM.


> +   3. Per-socket PSR features information structure
> +
> +      ```
> +      struct psr_socket_info {
> +          bool feat_init;
> +          struct feat_node *features[PSR_SOCKET_FEAT_NUM];
> +          spinlock_t ref_lock;
> +          unsigned int cos_ref[MAX_COS_REG_CNT];
> +          DECLARE_BITMAP(dom_ids, DOMID_IDLE + 1);
> +      };
> +      ```
> +
> +      We collect all PSR allocation features information of a socket in this
> +      `struct psr_socket_info`.
> +
> +      - Member `feat_init`
> +
> +        `feat_init` is a flag, to indicate whether the CPU init on a socket
> +        has been done.
> +
> +      - Member `features`
> +
> +        `features` is a pointer array to save all enabled features poniters
> +        according to feature position defined in `enum psr_feat_type`.
> +
> +      - Member `ref_lock`
> +
> +        `ref_lock` is a spin lock to protect `cos_ref`.
> +
> +      - Member `cos_ref`
> +
> +        `cos_ref` is an array which maintains the reference of one COS. It 
> maps
> +        to cos_reg_val[MAX_COS_REG_NUM] in `struct feat_node`. If one COS is
> +        used by one domain, the corresponding reference will increase by 
> one. If
> +        a domain releases the COS, the reference will decrease by one. The 
> array
> +        is indexed by COS ID.
> +
> +      - Member `dom_ids`
> +
> +        `dom_ids` is a bitmap, every bit corresponds to a domain. Index is
> +        domain_id. It is used to help restore domain_id to 0 when a socket is
> +        offline and then online again.

Did you mean "it is used to help restore domain_id to COS0, which has
all 1s in CBM, when a socket is offline and then online again."

Best,

Meng

-----------
Meng Xu
PhD Candidate in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel