Re: [Xen-devel] Load increase after memory upgrade (part2)

[Carsten Schiers <carsten@xxxxxxxxxx>]

I got the values in DomU. I will have

  - aprox. 5% load in DomU with 2.6.34 Xenified Kernel
  - aprox. 15% load in DomU with 2.6.32.46 Jeremy or 3.1.2 Kernel with one card 
attached
  - aprox. 30% load in DomU with 2.6.32.46 Jeremy or 3.1.2 Kernel with two 
cards attached

I looked through my old mails from you and you explained already the necessity 
of double
bounce buffering (PCI->below 4GB->above 4GB). What I don't understand is: why 
does the
Xenified kernel not have this kind of issue?

The driver in question is nearly identical between the two kernel versions. It 
is in
Drivers/media/dvb/ttpci by the way and when I understood the code right, the 
allo in 
question is:

        /* allocate and init buffers */
        av7110->debi_virt = pci_alloc_consistent(pdev, 8192, &av7110->debi_bus);
        if (!av7110->debi_virt)
                goto err_saa71466_vfree_4;

isn't it? I think the cards are constantly transferring the stream received 
through DMA. 

I have set dom0_mem=512M by the way, shall I change that in some way?

I can try out some things, if you want me to. But I have no idea what to do and 
where to
start, so I rely on your help...

Carsten.

-----Ursprüngliche Nachricht-----
Von: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] Im Auftrag von Konrad Rzeszutek 
Wilk
Gesendet: Freitag, 25. November 2011 19:43
An: Carsten Schiers
Cc: xen-devel; konrad.wilk
Betreff: Re: [Xen-devel] Load increase after memory upgrade (part2)

On Thu, Nov 24, 2011 at 01:28:44PM +0100, Carsten Schiers wrote:
> Hello again, I would like to come back to that thing...sorry that I did not 
> have the time up to now.
> 
> ??
> We (now) speak about
> 
> ??
> *     Xen 4.1.2
> *     Dom0 is Jeremy's 2.6.32.46 64 bit
> *     DomU in question is now 3.1.2 64 bit
> *     Same thing if DomU is also 2.6.32.46
> *     DomU owns two PCI cards (DVB-C) that o DMA
> *     Machine has 8GB, Dom0 pinned at 512MB
> 
> ??
> As compared to 2.6.34 Kernel with backported patches, the load on the DomU is 
> at least twice as high. It
> 
> will be "close to normal" if I reduce the memory used to 4GB.

That is in the dom0 or just in general on the machine?
> 
> ??
> As you can see from the attachment, you once had an idea. So should we try to 
> find something...?

I think that was to instrument swiotlb to give an idea of how
often it is called and basically have a matrix of its load. And
from there figure out if the issue is that:

 1). The drivers allocoate/bounce/deallocate buffers on every interrupt
    (bad, driver should be using some form of dma pool and most of the
    ivtv do that)

 2). The buffers allocated to the drivers are above the 4GB and we end
    up bouncing it needlessly. That can happen if the dom0 has most of
    the precious memory under 4GB. However, that is usually not the case
    as the domain isusually allocated from the top of the memory. The
    fix for that was to set dom0_mem=max:XX. .. but with Dom0 kernels
    before 3.1, the parameter would be ignored, so you had to use
    'mem=XX' on the Linux command line as well.

 3). Where did you get the load values? Was it dom0? or domU?



> 
> ??
> Carsten.
> ??
> -----Urspr??ngliche Nachricht-----
> An:konrad.wilk <konrad.wilk@xxxxxxxxxx>; 
> CC:linux <linux@xxxxxxxxxxxxxx>; xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>; 
> Von:Carsten Schiers <carsten@xxxxxxxxxx>
> Gesendet:Mi 29.06.2011 23:17
> Betreff:AW: Re: Re: Re: AW: Re: [Xen-devel] AW: Load increase after memory 
> upgrade?
> > Lets first do the c) experiment as that will likely explain your load 
> > average increase.
> ...
> > >c). If you want to see if the fault here lies in the bounce buffer 
> > being used more
> > >often in the DomU b/c you have 8GB of memory now and you end up using 
> > more pages
> > >past 4GB (in DomU), I can cook up a patch to figure this out. But an 
> > easier way is
> > >to just do (on the Xen hypervisor line): mem=4G and that will make 
> > think you only have
> > >4GB of physical RAM. ??If the load comes back to the normal "amount" 
> > then the likely
> > >culprit is that and we can think on how to fix this.
> 
> You are on the right track. Load was going down to "normal" 10% when reducing
> Xen to 4GB by the parameter. Load seems to be still a little, little bit lower
> with Xenified Kernel (8-9%), but this is drastically lower than the 20% we had
> before.

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel