[Xen-users] RE: Xen Network Performance

[Debasish Bose <dbose@xxxxxxxxx>]

I'm not at all interested in domU as I've figured out dom0-dom0 has similar 
values of throughput. So netfront stuff is not the culprit.

-Thanks
Deb
________________________________________
From: Debasish Bose
Sent: Wednesday, June 30, 2010 7:56 PM
To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Xen Network Performance

All,
           With some help I'm able to run Xen 4.0.1 rc3 and bring up the domUs. 
Earlier in a local network (Cisco switched), I've measured 114 MB/s network 
throughput (Rx side) using a single port of a Gbit card. Now with a domU-domU 
test, I'm only getting around 30 MB/s. (2.6.32.15 kernel, 4.0.1-rc3 Xen). I'm 
yet to run Xenoprof. Do you guys suggest any obvious thing I might check like,

    1. CPU / SMP scalability config. options in dom0 or any other kernel config 
option in that matter.

    2. Network driver (igb)
            lsmod | grep igb
               igb                    82854  0
               dca                     5816  1 igb
           Does that mean igb is not being used.

    3.  lsmod ...
              xen_netback           105755  0 [permanent]

        Probably I had compiled them statically. Is that bad or could be the 
reason?

    4. Earlier bare-machine test
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 172.16.30.206
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.14 real seconds = 113394.28 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.07, calls/sec = 14174.29
ttcp-t: 0.0user 0.0sys 0:00real 7% 0i+0d 370maxrss 0+2pf 317+0csw

Rx:
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.15 real seconds = 112929.24 KB/sec +++ := 110 MB/s 
= 880 Mb/s
ttcp-r: 6157 I/O calls, msec/call = 0.02, calls/sec = 42438.07
ttcp-r: 0.0user 0.1sys 0:00real 85% 0i+0d 382maxrss 0+2pf 5577+1csw

Now,
Tx:
root@EC2-TEST-2:/LinuxTTCP# ./ttcp -t 172.16.30.207
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp  -> 172.16.30.207
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.71 real seconds = 23092.58 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.35, calls/sec = 2886.57
ttcp-t: 0.0user 0.7sys 0:00real 98% 0i+0d 416maxrss 0+2pf 3+3csw

Rx:
root@EC1-TEST-1:/LinuxTTCP# ./ttcp -r
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001  tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.71 real seconds = 23055.21 KB/sec +++ 
<<<<<<<<<<<<<< BAD
ttcp-r: 2890 I/O calls, msec/call = 0.25, calls/sec = 4066.75
ttcp-r: 0.0user 0.3sys 0:00real 52% 0i+0d 440maxrss 0+2pf 2283+1csw

As you can see [ 42438.07 ] VS [ 4066.75 ] calls/sec probably indicates 
hyp/dom0 kernel is the culprit. Any early pointers is greatly appreciated.

-Thanks
Deb


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users