[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] TSQ accounting skb->truesize degrades throughput for large packets
On Tue, 2013-09-10 at 15:45 +0800, Jason Wang wrote: > For example, virtio-net will stop the tx queue when it finds the tx > queue may full and enable the queue when some packets were sent. In this > case, tsq works and throttles the total bytes queued in qdisc. This > usually happen during heavy network load such as two sessions of netperf. You told me skb were _orphaned_. This automatically _disables_ TSQ, after packets leave Qdisc. So you have a problem because your skb orphaning is only working when packets leave Qdisc. If you cant afford sockets being throttled, make sure you have no Qdisc ! > We notice a regression, and bisect shows it was introduced by TSQ. You do realize TSQ is a balance between throughput and latencies ? In case of TSQ, it was very clear that limiting amount of outstanding bytes in queues could have an impact on bandwidth. Pushing Megabytes of TCP packets with identical TCP timestamps is bad, because it prevents us doing delay based congestion control and a single flow could fill the Qdisc with a thousand of packets. (Self induced delays, see BufferBloat discussions) One known problem in TCP stack is that sendmsg() locks the socket for the duration of the call. sendpage() do not have this problem. tcp_tsq_handler() is deferred if tcp_tasklet_func() finds a locked socket. The owner of socket will call tcp_tsq_handler() when socket is released. So if you use sendmsg() with large buffers or if copyin data from user land involves page faults, it may explain why you need larger number of in-flight bytes to sustain a given throughput. You could take a look at commit c9bee3b7fdecb0c1d070c ("tcp: TCP_NOTSENT_LOWAT socket option"), and play with /proc/sys/net/ipv4/tcp_notsent_lowat, to force sendmsg() to release the socket lock every hundreds of kbytes. _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |