Re: [Xen-devel] Load increase after memory upgrade (part2)

[Konrad Rzeszutek Wilk <konrad@xxxxxxxxxx>]

On Thu, Nov 24, 2011 at 01:28:44PM +0100, Carsten Schiers wrote:
> Hello again, I would like to come back to that thing...sorry that I did not 
> have the time up to now.
> 
> ??
> We (now) speak about
> 
> ??
> *     Xen 4.1.2
> *     Dom0 is Jeremy's 2.6.32.46 64 bit
> *     DomU in question is now 3.1.2 64 bit
> *     Same thing if DomU is also 2.6.32.46
> *     DomU owns two PCI cards (DVB-C) that o DMA
> *     Machine has 8GB, Dom0 pinned at 512MB
> 
> ??
> As compared to 2.6.34 Kernel with backported patches, the load on the DomU is 
> at least twice as high. It
> 
> will be "close to normal" if I reduce the memory used to 4GB.

That is in the dom0 or just in general on the machine?
> 
> ??
> As you can see from the attachment, you once had an idea. So should we try to 
> find something...?

I think that was to instrument swiotlb to give an idea of how
often it is called and basically have a matrix of its load. And
from there figure out if the issue is that:

 1). The drivers allocoate/bounce/deallocate buffers on every interrupt
    (bad, driver should be using some form of dma pool and most of the
    ivtv do that)

 2). The buffers allocated to the drivers are above the 4GB and we end
    up bouncing it needlessly. That can happen if the dom0 has most of
    the precious memory under 4GB. However, that is usually not the case
    as the domain isusually allocated from the top of the memory. The
    fix for that was to set dom0_mem=max:XX. .. but with Dom0 kernels
    before 3.1, the parameter would be ignored, so you had to use
    'mem=XX' on the Linux command line as well.

 3). Where did you get the load values? Was it dom0? or domU?



> 
> ??
> Carsten.
> ??
> -----Urspr??ngliche Nachricht-----
> An:konrad.wilk <konrad.wilk@xxxxxxxxxx>; 
> CC:linux <linux@xxxxxxxxxxxxxx>; xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>; 
> Von:Carsten Schiers <carsten@xxxxxxxxxx>
> Gesendet:Mi 29.06.2011 23:17
> Betreff:AW: Re: Re: Re: AW: Re: [Xen-devel] AW: Load increase after memory 
> upgrade?
> > Lets first do the c) experiment as that will likely explain your load 
> > average increase.
> ...
> > >c). If you want to see if the fault here lies in the bounce buffer 
> > being used more
> > >often in the DomU b/c you have 8GB of memory now and you end up using 
> > more pages
> > >past 4GB (in DomU), I can cook up a patch to figure this out. But an 
> > easier way is
> > >to just do (on the Xen hypervisor line): mem=4G and that will make 
> > think you only have
> > >4GB of physical RAM. ??If the load comes back to the normal "amount" 
> > then the likely
> > >culprit is that and we can think on how to fix this.
> 
> You are on the right track. Load was going down to "normal" 10% when reducing
> Xen to 4GB by the parameter. Load seems to be still a little, little bit lower
> with Xenified Kernel (8-9%), but this is drastically lower than the 20% we had
> before.

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel