Re: [Xen-ia64-devel] [RFC] grant table API clean up and performancetuning memo

[Isaku Yamahata <yamahata@xxxxxxxxxxxxx>]

On Wed, Apr 26, 2006 at 01:44:26PM +0200, Tristan Gingold wrote:
> Le Mardi 28 Mars 2006 15:04, Tian, Kevin a écrit :
> > From: Isaku Yamahata [mailto:yamahata@xxxxxxxxxxxxx]
> >
> > >Sent: 2006年3月28日 20:27
> > >
> > >Without tracking a virtual address corresponding to a granted page,
> > >xen/ia64 have to flush all tlb/vhpt.
> > >I.e. xen/ia64 has to do something like
> > >    vhpt_flush();
> > >    flush_tlb_all();
> > >Here vhpt_flush() flushes the whole of vhpt table.
> > >
> > >On the other hand, dom0 knows the virtual address so
> > >dom0 may issues ptc.ga with page size (16KB by default).
> > >It results in vcpu_ptc_ga() of xen/ia64.
> > >It does
> > >    vhpt_flush_address(vadr,addr_range);
> > >    ia64_global_tlb_purge(vadr,vadr+addr_range,PAGE_SHIFT);
> > >Here vhpt flush range is 16kb.
> > >
> > >If xen/ia64 tracks a virtual address somehow,
> > >Xen/ia64 can flush vhpt with page size range so this become
> > >meaningless.
> >
> > See your point now. :-) So how about pass the virtual address prepared
> > for the granted page in host_addr at mapping time, though it's dom0 to
> > setup mapping right after(or even before) the grant mapping call? By
> > this way, though grant table doesn't setup the va mapping for guest, va is
> > recorded and later you can take that info to do more accurate flush.
> Two questions:
> It seems your mechanism still need to trust the domains.  So we need to add 
> checks for itc/ptc.
> 
> However, can a granted page be mapped several times ?
> If so, tracking is necessary and has a cost.

Grant table API allows it. Xen tracks it by the counter(act->pin).
However current xenLinux maps one-time per a granted page, I believe.


> From my point of view and my experience, flushing all the tlb/vhpt is really 
> costly (and even more with SMP-g), but it is necessary.

The cost is very high.


> Currently, we *don't* flush tlb/vhpt after unmapping granted page.  This 
> really creates instability (at least of gcc segfaults).

My understanding is as follows.
On the current P=M model, only dom0 maps/unmap granted pages.
Because Dom0 accesses a granted page via machine address (given that m=p),
flush isn't necessary.
But the bug is there, there must be something I miss.


> I don't have yet a magic solution.  I think we need to fix gcc segfault bug 
> first, and then we need to improve performance.

I fully agree with you. Bug fix first, tuning next.

-- 
yamahata

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel