Re: [Xen-devel] Xen & I/O in clusters - problems! New Information

[Håvard Bjerke <Havard.Bjerke@xxxxxxxxxxx>]

On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote:
> > 
> > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre 
> > results between
> > two xen dom0 nodes. I am not sure if these interrupts tells anything:
> 
> It's the different rate at which the eth0 interrupt counts go up
> during your bandwidth tests that is interesting. e.g. poll it
> once a second during the test.
> 

We tried sending 1 MB and measured:
non-SMP native Linux:
        ~ 130k interrupts
        114 kB/s
native Linux with compiled-in SMP support, single CPU:
        ~ 140k interrupts
        114 kB/s
Xen0:
        ~ 180k interrupts
        80 kB/s

> It looks like your native Linux is not using legacy PIC mode
> rather than using the ioapic. Have you tried an SMP native
> kernel to see if it gets the same interrupt layout as Xen?
>  

Layout in native linux w/o SMP:

           CPU0       
  0:  130710161          XT-PIC  timer
  1:          2          XT-PIC  keyboard
  2:          0          XT-PIC  cascade
  3:   12872557          XT-PIC  eth0
  7:          0          XT-PIC  ehci-hcd
  8:          1          XT-PIC  rtc
 10:          3          XT-PIC  usb-uhci
 11:     445263          XT-PIC  aic7xxx, aic7xxx, usb-uhci
 14:          1          XT-PIC  ide0
 15:          0          XT-PIC  libata
NMI:          0 
ERR:          0


Layout in native linux compiled with SMP:

           CPU0       
  0:     121362    IO-APIC-edge  timer
  1:          2    IO-APIC-edge  keyboard
  2:          0          XT-PIC  cascade
 14:          5    IO-APIC-edge  ide0
 16:          0   IO-APIC-level  usb-uhci
 18:     177583   IO-APIC-level  eth0
 19:          0   IO-APIC-level  usb-uhci
 24:       7587   IO-APIC-level  aic7xxx
 25:         30   IO-APIC-level  aic7xxx
NMI:          0 
LOC:     121307 
ERR:          0
MIS:          0


Layout in Xen0:

           CPU0       
  1:          2        Phys-irq  keyboard
 14:          3        Phys-irq  ide0
 18:     267274        Phys-irq  eth0
 24:      15910        Phys-irq  aic7xxx
 25:         30        Phys-irq  aic7xxx
128:          1     Dynamic-irq  misdirect
129:          0     Dynamic-irq  ctrl-if
130:     771917     Dynamic-irq  timer
131:          0     Dynamic-irq  timer_dbg, net-be-dbg
132:          0     Dynamic-irq  console
NMI:          0 
ERR:          0


Questions:
Do you think interrupt sharing is the problem?
Or the use of IO-APIC?
Is this an inherent problem with Xen?
Is it possible to change the interrupt scheme in Xen in order to achieve the 
same performance as in native linux?


Cheers,
Håvard


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_idU88&alloc_id065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel