[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH v2 0/5] mitigate the per-pCPU blocking list may be too long

VT-d PI introduces a per-pCPU blocking list to track the blocked vCPU
running on the pCPU. Theoretically, there are 32K domain on single
host, 128 vCPUs per domain. If all vCPUs are blocked on the same pCPU,
4M vCPUs are in the same list. Traversing this list consumes too
much time. More discussion can be found in [1,2,3].

To mitigate this issue, this series implements the following two methods:
1. put vcpus to another pcpu's list when the local pcpu's list length
reachs an upper bound. The upper bound is determined by total vcpu
count and total pcpu count in the system.
2. Don't put the blocked vCPUs which won't be woken by the wakeup
interrupt into the list.

PATCH 1/5 tracks the event, adding entry to PI blocking list. With the
patch, some data can be acquired to help to validate the following

PATCH 2/5 uses a global variable to track how many hvm vcpus on this
system. It is used to calculate the number limit of blocked vcpu on a
given pcpu.

In patch 3/5, a policy is used to restrict the vcpu count on a given
pcpu's pi blocking list in case the list grows too long.

Patch 4/5 adds a refcount to vcpu's pi_desc. If the pi_desc is
recorded in one IRTE, the refcount increases by 1 and If the pi_desc is
cleared in one IRTE, the refcount decreases by 1.

In Patch 5/5, one vCPU is added to PI blocking list only if its
pi_desc is referred by at least one IRTE.


Chao Gao (5):
  xentrace: add TRC_HVM_PI_LIST_ADD
  vcpu: track hvm vcpu number on the system
  VT-d PI: restrict the vcpu number on a given pcpu
  VT-d PI: Adding reference count to pi_desc
  VT-d PI: Don't add vCPU to PI blocking list for a case

 tools/xentrace/formats                 |   1 +
 xen/arch/x86/hvm/vmx/vmx.c             | 133 ++++++++++++++++++++++++++++++---
 xen/common/domain.c                    |   8 ++
 xen/drivers/passthrough/io.c           |   2 +-
 xen/drivers/passthrough/vtd/intremap.c |  59 ++++++++++++++-
 xen/include/asm-x86/hvm/domain.h       |   6 ++
 xen/include/asm-x86/hvm/trace.h        |   1 +
 xen/include/asm-x86/hvm/vmx/vmcs.h     |   3 +
 xen/include/asm-x86/iommu.h            |   2 +-
 xen/include/asm-x86/msi.h              |   2 +-
 xen/include/public/trace.h             |   1 +
 xen/include/xen/sched.h                |   2 +
 12 files changed, 203 insertions(+), 17 deletions(-)


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.