Re: [Xen-devel] large packet support in netfront driver and guest network throughput

[Anirban Chakraborty <abchak@xxxxxxxxxxx>]

On Sep 17, 2013, at 7:28 PM, annie li <annie.li@xxxxxxxxxx> wrote:

> 
> On 2013-9-18 1:53, Anirban Chakraborty wrote:
>> 
>> On 9/17/13 1:25 AM, "Wei Liu" <wei.liu2@xxxxxxxxxx> wrote:
>> 
>>> On Tue, Sep 17, 2013 at 10:09:21AM +0800, annie li wrote:
>>>> <snip>
>>>>>> I tried dom0 to dom0 and I got 9.4 Gbps, which is what I expected
>>>>>> (with GRO turned on in the physical interface). However, when I run
>>>>>> guest to guest, things fall off. Is large packet not supported in
>>>>>> netfront? I thought otherwise. I looked at the code and I do not see
>>>>>> any call to napi_gro_receive(), rather it is using
>>>>>> netif_receive_skb(). netback seems to be sending GSO packets to the
>>>>>> netfront, but it is being segmented to 1500 byte (as it appears from
>>>>>> the tcpdump).
>>>>>> 
>>>>> OK, I get your problem.
>>>>> 
>>>>> Indeed netfront doesn't make use of GRO API at the moment.
>>>> This is true.
>>>> But I am wondering why large packet is not segmented into mtu size
>>>> with upstream kernel? I did see large packets with upsteam kernel on
>>>> receive guest(test between 2 domus on same host).
>>>> 
>>> I think Anirban's setup is different. The traffic is from a DomU on
>>> another host.
>>> 
>>> I will need to setup testing environment with 10G link to test this.
>>> 
>>> Anirban, can you share your setup, especially DomU kernel version, are
>>> you using upstream kernel in DomU?
>> Sure..
>> I have two hosts, say h1 and h2 running XenServer 6.1.
>> h1 running Centos 6.4, 64bit kernel, say guest1 and h2 running identical
>> guest, guest2.
>> 
>> iperf server is running on guest1 with iperf client connecting from guest2.
>> 
>> I haven't tried with upstream kernel yet. However, what I found out is
>> that the netback on the receiving host is transmitting GSO segments to the
>> guest (guest1), but the packets are segmented at the netfront interface.
> 
> Did you try the guests on same host in your environment?

Yes, I did and it showed up 5.66 Gbps as opposed to 2.6-2.8Gbps.
> 
>> 
>> Annie's setup has both the guests running on the same host, in which case
>> packets are looped back.
> 
> If guests does not segment packets for same host case, it should not do 
> segment for different host case. For current upstream, Netback->netfront 
> mechanism does not treat differently for these two cases.

For the two guests (Centos 6.4, 2.6.32.-358) on the same host, the receiving 
guest indeed receives large packet, while this is not true if the receiving 
guest is on a different host. Since both the vifs are connected to the same 
linux bridge and hence packets passed between them are forwarded without 
hitting the wire. This explains the higher throughput. However, I would still 
expect segmented packet at the receiving guest, but certainly that is not what 
I am seeing.

Anirban



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel