Re: [Xen-devel] Logging External Page Writes for a Given Domain

[Anthony Liguori <aliguori@xxxxxxxxxx>]

Take a look at tools/libxc/xc_linux_save.c.  It does most of what you're 
looking for but in a different context (live migration of a domain).  
There is a paper, IIRC, that describes how migration in Xen works too 
that would be a good starter for understanding what that code's doing.
Regards,

Anthony Liguori

Scott Baker wrote:
> Hello, again!
>
> I'm back, with another question or two regarding logging external page 
> writes.  I was hoping that someone could point me to the proper source 
> files where I might make the following modifications:
> 1.   Enable dirty page tracking (in shadow paging mode), so I can know 
> which domain pages were modified.
> 2.   Reset dirty pages /right after/ the domain being logged is 
> stopped (before any other domain runs), so that I know which pages are 
> dirtied after the domain stops running.
> 3.   Dump (record) the contents of all dirty pages /right before/ the 
> domain being logged is about to start again.
> I've been poking around in the code base, but it's a little 
> overwhelming.  Another pointer in the right direction (e.g., 
> appropriate source files/locations, etc.) would be *very* helpful.  
> Also, a pointer to any general documentation on how Xen's shadow 
> paging and dirty page tracking work would be helpful as well.
> Oh, one last thing -- could these changes be made successfully on 
> versions of Xen >= 2.0; if not, what's the minimum/maximum version?
> Thank you all,
> Scott
>
> P.S.  The below is included to ensure the context of the request is 
> understood.
> Anthony Liguori wrote:
> > Scott Baker wrote:
> > > Anthony,
> > >
> > > Thanks for your response!  I've been thinking more about exactly 
> > > what the requirements are, and it amounts to the following.
> > > The end goal is to know the net result of any foreign-domain page 
> > > writes to all of a domain's (call it Domain P) pages.  I don't 
> > > actually need to know the contents of every individual foreign write 
> > > to Domain P's pages per se, just which the contents of the pages 
> > > that were modified by foreign domains since Domain P was last run.
> > > Let me give an example to illustrate.  Domain P is running.  It is 
> > > then suspended to allow another domain, Domain X, to start 
> > > execution.  Domain X happens to have a few pages of Domain P mapped 
> > > into its address space.  (This may be due to a device driver's page 
> > > mapping from Domain 0, or just another Domain mapping -- I don't 
> > > care why the pages are mapped).  Now, during its time slice, Domain 
> > > X modifies (over a period of several writes) two of Domain P's 
> > > pages: page 19 and 94.  Then, Domain X is suspended and Domain Y is 
> > > run.  Domain Y also modifies Domain P's page 94.  Finally, Domain Y 
> > > is suspended so Domain P can resume.  It is at this point, *just 
> > > before Domain P resumes*, that I would like to be able to have a 
> > > list of the pages (if any) that have been written by a foreign 
> > > domain (94 and 19, in this example).  With that list, I would like 
> > > to log the *entire contents* of those modified pages of Domain P for 
> > > my later use -- I don't need to know who wrote what when, just which 
> > > pages were modified so I can log their new contents.
> > Are you ignoring the SMP case (where two domains may be executing 
> > simultaneously)?  If so, I think the most direct approach would be to 
> > enable the dirty page tracking in shadow paging mode, and then dump 
> > any share pages before world switching that are dirty.  It still 
> > requires hypervisor modification, but it's reasonably straight forward.
> > Regards,
> >
> > Anthony Liguori
> > > So, in essence, *just before* every time that Domain P is allowed to 
> > > resume, I want to have a list of pages that have been modified 
> > > *since Domain P finished its previous time slice*, for the purpose 
> > > of logging the contents of each of those changed pages.
> > > Having said all that, what is the most straightforward way to 
> > > implement this (it needn't be the most efficient, though gross 
> > > inefficiency would be nice to avoid)?
> > > I really appreciate the feedback I've received.
> > > Thanks again,
> > >
> > > Scott
> > >
> > > Anthony Liguori wrote:
> > > > Scott Baker wrote:
> > > > > Hello, all:
> > > > >
> > > > > My team's goal is to be able to log all writes that are made to 
> > > > > any memory page of a certain domain, except those writes that the 
> > > > > domain itself makes.  That is to say, if Domain 2 is the domain we 
> > > > > want to log page writes for (where logging is capturing what was 
> > > > > written and its location), then we want to be able to log all the 
> > > > > writes made by any domain /except/ Domain 2 -- i.e., writes made 
> > > > > to shared pages that belong to Domain 2.
> > > > The details here are going to make a big difference.  Do you want 
> > > > to know the content of every write?
> > > > You'll have to modify Xen.  You can probably reuse some of the 
> > > > shadow paging code to track the dirty mappings of foreign pages for 
> > > > a domain.  However, this won't track the contents of the write.
> > > > If you want to do that, you're going to have to implement a large 
> > > > amount of emulation to track what data gets written to the page so 
> > > > you can emulate the writes completely.  If this is an important 
> > > > requirement, you may wish to try to user an emulator (Bochs or qemu).
> > > > Regards,
> > > >
> > > > Anthony Liguori
> > > > > Ideally, we would like to be able to have these writes for domain 
> > > > > /x/ detected and trap to a process running on Domain 0, with 
> > > > > minimal VMM modification.  Preferably, the method used would only 
> > > > > cause significant overhead when a write is made from outside 
> > > > > domain /x/ (rather than for every write /x/ and everyone else 
> > > > > makes), but I'm not picky about efficiency at the moment.
> > > > > The rough idea we have at this point is to make the monitor 
> > > > > process on Domain 0 mark all the pages of Domain /x/ as read-only 
> > > > > (while remembering which are actually read-only).  Then, when a 
> > > > > write-fault occurred, the VMM would pass it on to the monitoring 
> > > > > process, which would then let Domain /x/ finish the write, the 
> > > > > monitor would record what was written, and let everything continue 
> > > > > as normal.
> > > > > Unfortunately, I'm not sure how that vague sketch fits into Xen.  
> > > > > For the page table read-only flag setting, would we use the 
> > > > > update_va_mapping() hypercall?  And, how would the monitoring 
> > > > > process let Domain /x/ finish the write and then get control 
> > > > > back?  (Or, does the VMM know what is about to be written, so we 
> > > > > could just pass that to the monitoring process?)  Finally, where 
> > > > > in Xen's code would we have to go to modify the fault-handling 
> > > > > behavior so the callback could be made?
> > > > > Hope I'm clear, and hope you can help a newbie!
> > > > >
> > > > > Thanks,
> > > > > Scott
> > > > >
> > > > > _______________________________________________
> > > > > Xen-devel mailing list
> > > > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > > > http://lists.xensource.com/xen-devel
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > > http://lists.xensource.com/xen-devel
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel