[Xen-devel] [PATCH v3 0/4] sched: credit2: introduce per-vcpu hard and soft affinity

["Justin T. Weaver" <jtweaver@xxxxxxxxxx>]

Hello,

The credit2 vcpu scheduler currently ignores per-vcpu hard and soft affinity
masks.

The first patch updates the scheduler to ensure that vcpus only run
on pcpus on which they are allowed to run (hard affinity). I tested it using
xl vcpu-pin and xl vcpu-list. I changed the affinity in different ways using
scripted calls to vcpu-pin and observed the results using vcpu-list. Each VCPU
ran where it was supposed to.

Patch two factors out code from the credit scheduler (sched_credit.c) related
to soft affinity load balancing and places it in a common header (sched-if.h).
This allows credit2 to reuse the functions and defines in the soft affinity
patch (patch four of this series).

The third patch only indents some code in sched_credit2.c in order to make
reviewing patch four easier.

The fourth patch updates the scheduler to make more informed run queue and
pcpu decisions by considering which pcpus that vcpus prefer to run on (soft
affinity).

I tested this series on a NUMA machine with Dario Faggioli's "fix
per-socket runqueue setup" patch series applied. Without it, the credit2
scheduler only creates one run queue, regardless of the type of machine.

Here are the results I gathered from testing. Each guest had 2 vcpus and 1GB
of memory. The hardware consisted of two quad core Intel Xeon X5570 processors
and 8GB of RAM per node. The sysbench memory test was run with the num-threads
option set to four, and was run simultaneously on two, then six, then ten VMs.
Each result below is an average of three runs.

-------------------------------------------------------
| Sysbench memory, throughput MB/s (higher is better) |
-------------------------------------------------------
| #VMs |  No affinity  |   Pinning  | NUMA scheduling |
|   2  |    417.01     |    406.16  |     428.83      |
|   6  |    389.31     |    407.07  |     402.90      |
|  10  |    317.91     |    320.53  |     321.98      |
-------------------------------------------------------

Despite the overhead added, NUMA scheduling performed best in both the two and
ten VM tests.

Thank you for all the review comments!

Justin Weaver
Masters candidate
University of HawaiÊi at MÄnoa

---
[1/4] sched: credit2: respect per-vcpu hard affinity
[2/4] sched: factor out per-vcpu affinity related code to common header file
[3/4] sched: credit2: indent code sections to make review of patch 4/4 easier
[4/4] sched: credit2: consider per-vcpu soft affinity

xen/common/sched_credit.c  |   87 +------
xen/common/sched_credit2.c |  551 ++++++++++++++++++++++++++++++++++++--------
xen/include/xen/sched-if.h |   65 ++++++
3 files changed, 536 insertions(+), 167 deletions(-)

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel