[PATCH v2 0/4] x86/HVM: batch the flushes of grant unmaps

[Luca Di Maio <luca@xxxxxxxxx>]

Hi all,

GNTTABOP_unmap_grant_ref already batches the TLB flushes of a PV
caller: gnttab_unmap_grant_ref() unmaps up to 32 grants, flushes the
TLBs once with gnttab_flush_tlb(), and only then releases the pages.
gnttab_flush_tlb() does nothing for a translated caller, whose unmaps
go through the p2m, and the p2m code flushes the guest TLBs and the
IOTLB for every page instead. A PVH dom0 running netback pays both
flushes for every page a domU transmits.

This series gives translated callers the same batching, for both
flushes, before the pages are released. The IOTLB half has no PV
counterpart: a PV dom0's IOMMU maps all of RAM by default, so its
grant unmaps don't touch it. Patches 1 and 2 are preparatory, and
patches 3 and 4 have the details.

iperf3 between a PVH dom0 running netback and two PVH domUs:

  machine:  c7i.metal-24xl, Xeon Platinum 8488C, 96 threads,
            VT-d with shared EPT tables
  dom0:     96 pinned vCPUs, 4 GiB, Debian 13, Linux 6.12
  domUs:    8 vCPUs, 1 GiB, Linux 6.18, on a bridge
  Xen:      staging at 9a6cc49b44, built with GCC 14.2
  runs:     10 s, Gbit/s received, higher is better, 60 per row
            from 4 boots of each build, interleaved
  shown:    mean of the runs +/- their relative standard deviation

                   base          patched       %change
  domU to dom0
    1 stream       0.64 +/-1%    3.11 +/-2%      +386%
    2 streams      0.81 +/-6%    3.16 +/-2%      +291%
    4 streams      0.84 +/-4%    2.69 +/-7%      +221%
    8 streams      0.83 +/-2%    2.40 +/-5%      +190%
    16 streams     0.81 +/-2%    2.23 +/-2%      +175%
  domU to domU
    1 stream       0.62 +/-6%    2.29 +/-5%      +267%
    2 streams      0.73 +/-7%    2.26 +/-10%     +211%
    4 streams      0.75 +/-4%    1.86 +/-16%     +150%
    8 streams      0.73 +/-4%    1.73 +/-9%      +136%
    16 streams     0.71 +/-3%    1.62 +/-6%      +128%
  dom0 to domU
    1 stream       4.19 +/-6%    3.93 +/-8%      -6.2%
    2 streams      4.87 +/-9%    4.68 +/-11%     -4.0%
    4 streams      8.73 +/-21%   8.34 +/-19%     -4.5%
    8 streams     12.53 +/-17%  13.05 +/-17%     +4.1%
    16 streams    16.85 +/-12%  16.40 +/-16%     -2.7%

The table comes from one of 13 sessions on this instance type, which
ran 7,779 runs over 57 boots in total.

A domU transmits 2.2 to 4.8 times as much, and 2.2 to 4.0 times as
much per busy dom0 vCPU, where base never exceeds 0.84 Gbit/s whatever
the number of streams.

>From dom0 to a domU, netback fills the domU's buffers with grant
copies and maps few of its packets, so the series barely runs there
and the differences stay within the margin of error.

Between boots of the same build, the single-stream mean varied no
more than the run-to-run spread predicts. A debug build ran the same
benchmark without assertion failures or IOMMU faults.

Changes in v2:
- Patch 1 is new: VT-d rounds flush ranges to an aligned power of two
  itself, instead of the grant unmap batch doing it (Teddy).
- iommu_iotlb_flush() checks iommu_dont_flush_iotlb and records the
  range as well as the flags of the flushes it skips, instead of its
  callers recording the flags in a uint8_t (Teddy, Jan).
- The grant unmap batch flushes the recorded range, instead of the span
  of the unmapped gfns (Teddy).
- The table's patched column is now re-run against v2

Thanks,
L

Luca Di Maio (4):
  VT-d: round IOTLB flush ranges up to an aligned power of two
  IOMMU: record the flushes skipped under iommu_dont_flush_iotlb
  x86/HVM: defer IOTLB flushes across a grant unmap batch
  x86/HVM: defer p2m TLB flushes across a grant unmap batch

 xen/arch/arm/include/asm/grant_table.h     |  3 ++
 xen/arch/x86/hvm/grant_table.c             | 55 +++++++++++++++++++++-
 xen/arch/x86/include/asm/grant_table.h     | 13 +++++
 xen/arch/x86/include/asm/hvm/grant_table.h |  5 ++
 xen/arch/x86/include/asm/hvm/vcpu.h        | 10 ++++
 xen/arch/x86/mm/p2m-ept.c                  |  2 +-
 xen/common/grant_table.c                   |  6 +++
 xen/drivers/passthrough/iommu.c            | 17 ++++++-
 xen/drivers/passthrough/vtd/iommu.c        |  4 ++
 xen/include/xen/iommu.h                    |  6 +++
 10 files changed, 117 insertions(+), 4 deletions(-)


base-commit: 41ddcfe808ec1228289ae60c30768d50a58c98fe
-- 
2.47.3