[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] TSQ accounting skb->truesize degrades throughput for large packets
On Fri, Sep 06, 2013 at 05:57:48AM -0700, Eric Dumazet wrote: > On Fri, 2013-09-06 at 11:16 +0100, Wei Liu wrote: > > Hi Eric > > > > I have some questions regarding TSQ and I hope you can shed some light > > on this. > > > > Our observation is that with the default TSQ limit (128K), throughput > > for Xen network driver for large packets degrades. That's because we now > > only have 1 packet in queue. > > > > I double-checked that skb->len is indeed <64K. Then I discovered that > > TSQ actually accounts for skb->truesize and the packets generated had > > skb->truesize > 64K which effectively prevented us from putting 2 > > packets in queue. > > > > There seems to be no way to limit skb->truesize inside driver -- the skb > > is already constructed when it comes to xen-netfront. > > > > What is the skb->truesize value then ? It must be huge, and its clearly > a problem, because the tcp _receiver_ will also grow its window slower, > if packet is looped back. > It's ~66KB. > > My questions are: > > 1) I see the comment in tcp_output.c saying: "TSQ : sk_wmem_alloc > > accounts skb truesize, including skb overhead. But thats OK", I > > don't quite understand why it is OK. > > 2) presumably other drivers will suffer from this as well, is it > > possible to account for skb->len instead of skb->truesize? > > Well, I have no problem to get line rate on 20Gb with a single flow, so > other drivers have no problem. > OK, good to know this. > > 3) if accounting skb->truesize is on purpose, does that mean we only > > need to tune that value instead of trying to fix our driver (if > > there is a way to)? > > The check in TCP allows for two packets at least, unless a single skb > truesize is 128K ? > > > if (atomic_read(&sk->sk_wmem_alloc) >= sysctl_tcp_limit_output_bytes) { > set_bit(TSQ_THROTTLED, &tp->tsq_flags); > break; > } > > So if a skb->truesize is 100K, this condition allows two packets, before > throttling the third packet. > OK. I need to check why we're getting only 1 then. Thanks for your reply. Wei. > Its actually hard to account for skb->len, because sk_wmem_alloc > accounts for skb->truesize : I do not want to add another > sk->sk_wbytes_alloc new atomic field. > > _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |