[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] PML (Page Modification Logging) design for Xen

On 02/12/2015 03:08 PM, Tian, Kevin wrote:
From: Kai Huang [mailto:kai.huang@xxxxxxxxxxxxxxx]
Sent: Thursday, February 12, 2015 2:46 PM

On 02/12/2015 02:25 PM, Tian, Kevin wrote:
From: Kai Huang [mailto:kai.huang@xxxxxxxxxxxxxxx]
Sent: Thursday, February 12, 2015 10:35 AM

On 02/11/2015 09:13 PM, Jan Beulich wrote:
On 11.02.15 at 12:52, <andrew.cooper3@xxxxxxxxxx> wrote:
On 11/02/15 08:28, Kai Huang wrote:
With PML, we don't have to use write protection but just clear D-bit
of EPT entry of guest memory to do dirty logging, with an additional
PML buffer full VMEXIT for 512 dirty GPAs. Theoretically, this can
reduce hypervisor overhead when guest is in dirty logging mode, and
therefore more CPU cycles can be allocated to guest, so it's expected
benchmarks in guest will have better performance comparing to
One issue with basic EPT A/D tracking was the scan of the EPT tables.
Here, hardware will give us a list of affected gfns, but how is Xen
supposed to efficiently clear the dirty bits again?  Using EPT
misconfiguration is no better than the existing fault path.
Why not? The misconfiguration exit ought to clear the D bit for all
511 entries in the L1 table (and set it for the one entry that is
currently serving the access). All further D bit handling will then
be PML based.
Indeed, we clear D-bit in EPT misconfiguration. In my understanding, the
sequences are as follows:

1) PML enabled for the domain.
2) ept_invalidate_emt (or ept_invalidate_emt_range) is called.
3) Guest accesses specific GPA (which has been invalidated by step 2),
and EPT misconfig is triggered.
4) Then resolve_misconfig is called, which fixes up GFN (above GPA >>
12) to p2m_ram_logdirty, and calls ept_p2m_type_to_flags, in which we
clear D-bit of EPT entry (instead of clear W-bit) if p2m type is
p2m_ram_logdirty. Then dirty logging of this GFN will be handled by PML.

The above 2) ~ 4) will be repeated when log-dirty radix tree is cleared.
is ept_invalidate_emt required by existing logdirty mode or by PML enable?
It's in existing logdirty code.
can we clear D bits directly when log-dirty radix tree is cleared to reduce
EPT misconfig exits for repeatedly dirtied pages?
Theoretically we can, and looks logdirty for video ram is done in this
way (logdirty for the page is re-enabled while it is reported to
dirty_bitmap). One thing is looks video ram logdirty only exists for HAP
But in current log dirty implementation for global logdirty, at common
paging layer, the log-dirty radix tree is cleaned in single step after
reporting all dirty pages to userspace. And it just calls
ept_invalidate_emt essentially. Therefore we need to modify logdirty
common code at paging layer to achieve this, which is more like logdirty
enhancement but not related to PML enabling directly. And any change of
interface in paging layer requires modification in shadow mode
accordingly, so currently I just choose not to do it.

for general log dirty, ept_invalidate_emt is required because there is
access permission change (dirtied page becomes rw after 1st fault,
so need to change them back to ro again for the new dirty tracking
round). But for PML, there's no permission change at all (always rw),
so such behavior should be noted by general logdirty layer for better
optimization. I'm OK not doing so for initial enabling patch, but it's
something you can think about later. :-)
Yes thanks for the point :)



Xen-devel mailing list

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.