Re: [Xen-devel] Load increase after memory upgrade (part2)

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2011-11-28 at 16:45 +0000, Konrad Rzeszutek Wilk wrote:
> On Mon, Nov 28, 2011 at 03:40:13PM +0000, Ian Campbell wrote:
> > On Mon, 2011-11-28 at 15:28 +0000, Konrad Rzeszutek Wilk wrote:
> > > On Fri, Nov 25, 2011 at 11:11:55PM +0100, Carsten Schiers wrote:
> > 
> > > > I looked through my old mails from you and you explained already the 
> > > > necessity of double
> > > > bounce buffering (PCI->below 4GB->above 4GB). What I don't understand 
> > > > is: why does the
> > > > Xenified kernel not have this kind of issue?
> > > 
> > > That is a puzzle. It should not. The code is very much the same - both
> > > use the generic SWIOTLB which has not changed for years.
> > 
> > The swiotlb-xen used by classic-xen kernels (which I assume is what
> > Carsten means by "Xenified") isn't exactly the same as the stuff in
> > mainline Linux, it's been heavily refactored for one thing. It's not
> > impossible that mainline is bouncing something it doesn't really need
> > to.
> 
> The usage, at least with 'pci_alloc_coherent' is that there is no bouncing
> being done. The alloc_coherent will allocate a nice page, underneath the 4GB
> mark and give it to the driver. The driver can use it as it wishes and there
> is no need to bounce buffer.

Oh, I didn't realise dma_alloc_coherent was part of swiotlb now. Only a
subset of swiotlb is in use then, all the bouncing stuff _should_ be
idle/unused -- but has that been confirmed?

> 
> But I can't find the implementation of that in the classic Xen-SWIOTLB. It 
> looks
> as if it is using map_single which would be taking the memory out of the
> pool for a very long time, instead of allocating memory and "swizzling" the 
> MFNs.
> [Note, I looked at the 2.6.18 hg tree for classic, the 2.6.34 is probably
> improved much better so let me check that]
> 
> Carsten, let me prep up a patch that will print some diagnostic information
> during the runtime - to see how often it does the bounce, the usage, etc..
> 
> > 
> > It's also possible that the dma mask of the device is different/wrong in
> > mainline leading to such additional bouncing.
> 
> If one were to use map_page and such - yes. But the alloc_coherent bypasses
> that and ends up allocating it right under the 4GB (or rather it allocates
> based on the dev->coherent_mask and swizzles the MFNs as required).
> 
> > 
> > I guess it's also possible that the classic-Xen kernels are playing fast
> > and loose by not bouncing something they should (although if so they
> > appear to be getting away with it...) or that there is some difference
> > which really means mainline needs to bounce while classic-Xen doesn't.
> 
> <nods> Could be very well.
> > 
> > Ian.
> > 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel