Re: [Xen-devel] Xen cluster n/w performance (again!)

[David Becker <becker@xxxxxxxxxxx>]

I looked closely at tcpdumps of an iperf stream flowing into domain-0 from a
stock linux box.

It looks like my xen0 is not able to send out ACK packets while receiving
incoming data packets. 
That is based on the first attached postscript graph.  It was generated 
by tcptrace/xplot from a tcpdump taken on xen0 while iperf data for xen0
is arriving.
  The green line plots the time of the highest seen ACK seq number.
  The yellow line plots the seq number of the window limit over time.
  The black segments show time of incoming data (diamonds show TCP PUSH flag)

The second postscript graph traces an iperf stream as seen from the sender side.
The sender runs stock linux 2.4.25.  The receiver runs 2.4.27-xen0.

It shows that the sender fills the window as soon as green acks arrive, then
waits quite a while for the next batch of acks to resume transmitting to
xen0.  Increasing the window size makes no difference as the sender keeps the
window full (graphwise that means the yellow and green lines are farther
apart, but the black segments reach the yellow line as soon as the acks
arrive).

Looking at iperf flows between stock linux boxes shows the sender never
fills the window (the black segments rarely reach the yellow line).

I'm guessing this means the handling of Rx interrupts completely blocks out
progress on the Tx side.  I tried throwing in a noapic kernel param but
it made no difference.

Attachment: xendst.ps
Description: PostScript document

Attachment: xensrc.ps
Description: PostScript document