[Xen-devel] [PATCH v3 0/3] mitigate the per-pCPU blocking list may be too long

[Chao Gao <chao.gao@xxxxxxxxx>]

VT-d PI introduces a per-pCPU blocking list to track the blocked vCPU
on a given pCPU. Theoretically, there are 32K domain on single host,
128 vCPUs per domain. If all vCPUs are blocked on the same pCPU,
4M vCPUs are in the same list. Traversing this list consumes too
much time. More discussion can be found in [1,2,3].

To mitigate this issue, this series put vcpus to another pcpu's list
when the local pcpu's list length reachs an upper bound. The upper
bound is determined by total vcpu count and total pcpu count in the
system.

PATCH 1/3 adds a counter to track the per-pCPU blocking list's length
and to aid analysis and obtain the list length, add some relevant
events to xentrace. With this patch, some data can be acquired to help
to validate patch 3/3. 

PATCH 2/3 uses a global variable to track how many hvm vcpus on this
system. It is used to calculate the number limit of blocked vcpu on a
given pcpu.

In patch 3/3, a policy is used to restrict the vcpu count on a given
pcpu's pi blocking list in case the list grows too long.

In this new version, the method only putting vcpus which are referred
by at least one IRTE to the list is removed for
1. The implementation is intrusive.
2. Now, no evidence shows its necessity.

[1] 
https://lists.gt.net/xen/devel/422661?search_string=VT-d%20posted-interrupt%20core%20logic%20handling;#422661
[2] 
https://lists.gt.net/xen/devel/422567?search_string=%20The%20length%20of%20the%20list%20depends;#422567
[3] 
https://lists.gt.net/xen/devel/472749?search_string=enable%20vt-d%20pi%20by%20default;#472749

Chao Gao (3):
  VT-d PI: track the vcpu number in pi blocking list
  vcpu: track hvm vcpu number on the system
  VT-d PI: restrict the vcpu number on a given pcpu

 tools/xentrace/formats          |   3 +
 tools/xentrace/xenalyze.c       | 159 +++++++++++++++++++++++++++++++++++++---
 xen/arch/x86/domain.c           |  10 +++
 xen/arch/x86/hvm/vmx/vmx.c      |  97 +++++++++++++++++++++---
 xen/include/asm-x86/domain.h    |   2 +
 xen/include/asm-x86/hvm/trace.h |   2 +
 xen/include/public/trace.h      |   6 ++
 7 files changed, 261 insertions(+), 18 deletions(-)

-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel