[Xen-devel] [PATCH v5 0/4] mitigate the per-pCPU blocking list may be too long

[Chao Gao <chao.gao@xxxxxxxxx>]

Changes in v5:
 - In patch 1, add check the sanity of vcpus count on pi blocking list 
   and also drop George's Reviewed-by.
 - In patch 3, introduce a new function to find proper pCPU to accept
 the blocked vcpu.
 - In patch 4, add support of tracking the operations on pi blocking list
 and generating scatterplot of pi list length

VT-d PI introduces a per-pCPU blocking list to track the blocked vCPU
on a given pCPU. Theoretically, there are 32K domain on single host,
128 vCPUs per domain. If all vCPUs are blocked on the same pCPU,
4M vCPUs are in the same list. Traversing this list consumes too
much time. More discussion can be found in [1,2,3].

To mitigate this issue, this series put vcpus to another pcpu's list
when the local pcpu's list length reachs an upper bound which is the
average vcpus per pcpu ratio plus a constant. 

PATCH 1/4 adds a counter to track the per-pCPU blocking list's length.

PATCH 2/4 uses a global variable to track how many hvm vcpus on this
system. It is used to calculate the average vcpus per pcpu ratio.

patch 3/4 employs a policy to restrict the vcpu count on a given
pcpu's pi blocking list in case the list grows too long. In one work,
If list length is smaller than the upper bound, the vcpu is added to
the pi blocking list of the pcpu which it is running on. Otherwise,
another online pcpu is chosen to accept the vcpu.

patch 4/4 adds some relevant events to xentrace to aid analysis of
the list length. With this patch, some data can be acquired to
validate patch 3/4. 

[1] 
https://lists.gt.net/xen/devel/422661?search_string=VT-d%20posted-interrupt%20core%20logic%20handling;#422661
[2] 
https://lists.gt.net/xen/devel/422567?search_string=%20The%20length%20of%20the%20list%20depends;#422567
[3] 
https://lists.gt.net/xen/devel/472749?search_string=enable%20vt-d%20pi%20by%20default;#472749

Chao Gao (4):
  VT-d PI: track the number of vcpus on pi blocking list
  x86/vcpu: track hvm vcpu number on the system
  VT-d PI: restrict the number of vcpus in a given pcpu's PI blocking
    list
  xentrace: add support for HVM's PI blocking list operation

 tools/xentrace/formats        |   2 +
 tools/xentrace/xenalyze.c     | 116 +++++++++++++++++++++++++++++
 xen/arch/x86/hvm/hvm.c        |   6 ++
 xen/arch/x86/hvm/vmx/vmx.c    | 166 ++++++++++++++++++++++++++++++++++++------
 xen/include/asm-x86/hvm/hvm.h |   3 +
 xen/include/public/trace.h    |   5 ++
 6 files changed, 275 insertions(+), 23 deletions(-)

-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel