Re: [Xen-devel] Load increase after memory upgrade (part2)

[Carsten Schiers <carsten@xxxxxxxxxx>]

Thank you, Konrad.

I applied the patch to 3.1.2. In order to have a clear picture, I only enabled 
one PCI card.
The result is:

[   28.028032] Starting SWIOTLB debug thread.
[   28.028076] swiotlb_start_thread: Go!
[   28.028622] xen_swiotlb_start_thread: Go!
[   33.028153] 0 [budget_av 0000:00:00.0] bounce: from:555352(slow:0)to:0 
map:329 unmap:0 sync:555352
[   33.028294] SWIOTLB is 2% full
[   38.028178] 0 budget_av 0000:00:00.0 alloc coherent: 4, free: 0
[   38.028230] 0 [budget_av 0000:00:00.0] bounce: from:127981(slow:0)to:0 map:0 
unmap:0 sync:127981
[   38.028352] SWIOTLB is 2% full
[   43.028170] 0 [budget_av 0000:00:00.0] bounce: from:128310(slow:0)to:0 map:0 
unmap:0 sync:128310
[   43.028310] SWIOTLB is 2% full
[   48.028199] 0 [budget_av 0000:00:00.0] bounce: from:127981(slow:0)to:0 map:0 
unmap:0 sync:127981
[   48.028334] SWIOTLB is 2% full
[   53.028170] 0 [budget_av 0000:00:00.0] bounce: from:128310(slow:0)to:0 map:0 
unmap:0 sync:128310
[   53.028309] SWIOTLB is 2% full
[   58.028138] 0 [budget_av 0000:00:00.0] bounce: from:126994(slow:0)to:0 map:0 
unmap:0 sync:126994
[   58.028195] SWIOTLB is 2% full
[   63.028170] 0 [budget_av 0000:00:00.0] bounce: from:121401(slow:0)to:0 map:0 
unmap:0 sync:121401
[   63.029560] SWIOTLB is 2% full
[   68.028193] 0 [budget_av 0000:00:00.0] bounce: from:127981(slow:0)to:0 map:0 
unmap:0 sync:127981
[   68.028329] SWIOTLB is 2% full
[   73.028104] 0 [budget_av 0000:00:00.0] bounce: from:122717(slow:0)to:0 map:0 
unmap:0 sync:122717
[   73.028244] SWIOTLB is 2% full
[   78.028191] 0 [budget_av 0000:00:00.0] bounce: from:127981(slow:0)to:0 map:0 
unmap:0 sync:127981
[   78.028331] SWIOTLB is 2% full
[   83.028112] 0 [budget_av 0000:00:00.0] bounce: from:128310(slow:0)to:0 map:0 
unmap:0 sync:128310
[   83.028171] SWIOTLB is 2% full

Was that long enough? I hope this helps.

Carsten.

-----Ursprüngliche Nachricht-----
Von: Konrad Rzeszutek Wilk [mailto:konrad@xxxxxxxxxx] 
Gesendet: Freitag, 2. Dezember 2011 16:24
An: Konrad Rzeszutek Wilk
Cc: Ian Campbell; xen-devel; Carsten Schiers; zhenzhong.duan@xxxxxxxxxx; 
lersek@xxxxxxxxxx
Betreff: Re: [Xen-devel] Load increase after memory upgrade (part2)

> > > > > That is a puzzle. It should not. The code is very much the same - both
> > > > > use the generic SWIOTLB which has not changed for years.
> > > > 
> > > > The swiotlb-xen used by classic-xen kernels (which I assume is what
> > > > Carsten means by "Xenified") isn't exactly the same as the stuff in
> > > > mainline Linux, it's been heavily refactored for one thing. It's not
> > > > impossible that mainline is bouncing something it doesn't really need
> > > > to.
> > > 
> > > The usage, at least with 'pci_alloc_coherent' is that there is no bouncing
> > > being done. The alloc_coherent will allocate a nice page, underneath the 
> > > 4GB
> > > mark and give it to the driver. The driver can use it as it wishes and 
> > > there
> > > is no need to bounce buffer.
> > 
> > Oh, I didn't realise dma_alloc_coherent was part of swiotlb now. Only a
> > subset of swiotlb is in use then, all the bouncing stuff _should_ be
> > idle/unused -- but has that been confirmed?
> 
> Nope. I hope that the diagnostic patch I have in mind will prove/disprove 
> that.
> Now I just need to find a moment to write it :-)

Done!

Carsten, can you please patch your kernel with this hacky patch and
when you have booted the new kernel, just do

modprobe dump_swiotlb

it should give an idea of how many bounces are happening, coherent
allocations, syncs, and so on.. along with the last driver that
did those operations.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel