[Xen-devel] [PATCH v4 12/12] docs: add xl-psr.markdown

[Chao Peng <chao.p.peng@xxxxxxxxxxxxxxx>]

Add document to introduce basic concepts and terms in PSR family
techonologies and the xl/libxl interfaces.

Signed-off-by: Chao Peng <chao.p.peng@xxxxxxxxxxxxxxx>
---
 docs/man/xl.pod.1         |   7 +++
 docs/misc/xl-psr.markdown | 111 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 118 insertions(+)
 create mode 100644 docs/misc/xl-psr.markdown

diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index dfab921..b71d6e6 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -1472,6 +1472,9 @@ occupancy monitoring share the same set of underlying 
monitoring service. Once
 a domain is attached to the monitoring service, monitoring data can be showed
 for any of these monitoring types.
 
+See L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html> for more
+informations.
+
 =over 4
 
 =item B<psr-cmt-attach> [I<domain-id>]
@@ -1501,6 +1504,9 @@ applications. In Xen implementation, CAT is used to 
control cache allocation
 on VM basis. To enforce cache on a specific domain, just set capacity bitmasks
 (CBM) for the domain.
 
+See L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html> for more
+informations.
+
 =over 4
 
 =item B<psr-cat-cbm-set> [I<OPTIONS>] [I<domain-id>] [I<cbm>]
@@ -1546,6 +1552,7 @@ And the following documents on the xen.org website:
 L<http://xenbits.xen.org/docs/unstable/misc/xl-network-configuration.html>
 L<http://xenbits.xen.org/docs/unstable/misc/xl-disk-configuration.txt>
 L<http://xenbits.xen.org/docs/unstable/misc/xsm-flask.txt>
+L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>
 
 For systems that don't automatically bring CPU online:
 
diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
new file mode 100644
index 0000000..44f6f8c
--- /dev/null
+++ b/docs/misc/xl-psr.markdown
@@ -0,0 +1,111 @@
+# Intel Platform Shared Resource Monitoring/Control in xl/libxl
+
+This document introduces Intel Platform Shared Resource Monitoring/Control
+technologies, their basic concepts and the xl/libxl interfaces.
+
+## Cache Monitoring Technology (CMT)
+
+Cache Monitoring Technology (CMT) is a new feature available on Intel Haswell
+and later server platforms that allows an OS or Hypervisor/VMM to determine
+the usage of cache(currently only L3 cache supported) by applications running
+on the platform. A Resource Monitoring ID (RMID) is the abstraction of the
+application(s) that will be monitored for its cache usage. The CMT hardware
+tracks cache utilization of memory accesses according to the RMID and reports
+monitored data via a counter register.
+
+Detailed information please refer to Intel SDM chapter 17.14.
+
+In Xen's implementation, each domain in the system can be assigned a RMID
+independently, while RMID=0 is reserved for monitoring domains that doesn't
+enable CMT service. RMID is opaque for xl/libxl and is only used in
+hypervisor.
+
+### xl interfaces
+
+A domain is assigned a RMID implicitly by attaching it to CMT service:
+
+xl psr-cmt-attach domid
+
+After that, cache usage for the domain can be showed by:
+
+xl psr-cmt-show cache_occupancy <domid>
+
+Once monitoring is not needed any more, the domain can be detached from the
+CMT service by:
+
+xl psr-cmt-detach domid
+
+The attaching may fail because of no free RMID available. In such case
+unused RMID(s) can be freed by detaching corresponding domains from CMT
+services. Maximum COS number in the system can also be obtained by:
+
+xl psr_cmt-show
+
+## Memory Bandwidth Monitoring (MBM)
+
+Memory Bandwidth Monitoring(MBM) is a new hardware feature available on Intel
+Broadwell and later server platforms which builds on the CMT infrastructure to
+allow monitoring of system memory bandwidth. It introduces two new monitoring
+event type to monitor system total/local memory bandwidth. The same RMID can
+be used to monitor both cache usage and memory bandwidth at the same time.
+
+Detailed information please refer to Intel SDM chapter 17.14.
+
+In Xen's implementation, MBM shares the same set of underlying monitoring
+service with CMT and can be used to monitor memory bandwidth on domain basis.
+
+The xl/libxl interface is the same with that of CMT. The difference is the
+monitor type is corresponding memory monitoring type(local_mem_bandwidth/
+total_mem_bandwidth) but not cache_occupancy.
+
+## Cache Allocation Technology (CAT)
+
+Cache Allocation Technology (CAT) is a new feature available on Intel
+Broadwell and later server platforms that allows an OS or Hypervisor/VMM to
+partition cache allocation(i.e. L3 cache) based on application priority or
+Class of Service(COS). Each COS is configured using capacity bitmasks (CBM)
+which represent cache capacity and indicate the degree of overlap and
+isolation between classes. System cache resource is divided into numbers of
+minimum portions which is then made up into subset for cache partition. Each
+portion corresponds to a bit in CBM and the set bit represents the
+corresponding cache portion is available.
+
+Detailed information please refer to Intel SDM chapter 17.15.
+
+In Xen's implementation, CBM can be set/get with libxl/xl interfaces but COS
+is maintained in hypervisor only. The cache partition granularity is per
+domain, each domain has COS=0 assigned by default, the corresponding CBM is
+all-ones, which means all the cache resource can be used by default.
+
+### xl interfaces
+
+The simplest way to change a domain's CBM from its default is running:
+
+xl psr-cat-cbm-set  [OPTIONS] <domid> <cbm>
+
+where cbm is a decimal/hexadecimal number to represent the corresponding cache
+subset can be used.
+
+A cbm is valid only when:
+
+ * Set bits only exist in the range of [0, cbm_len), where cbm_len can be
+   obtained with 'xl psr-cat-show'.
+ * All the set bits is contiguous.
+ * Is not the same with the current cbm of the domain.
+
+In multi-sockets system, the same cbm will be set to each socket by default.
+Per socket cbm can be specified with '--socket SOCKET' option.
+
+The cbm may be not set successfully because of no enough COS available. In such
+case unused COS(es) may be freed by setting CBM of all related domains to its
+default value(all-ones).
+
+System CAT information(such as maximum COS and CBM length) and per domain CBM
+settings can be showed by:
+
+xl psr-cat-show
+
+## Reference
+
+[1] Intel SDM
+(http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html).
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel