[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] xennet: skb rides the rocket: 20 slots

On 2013-1-10 20:26, Ian Campbell wrote:
On Thu, 2013-01-10 at 11:22 +0000, ANNIE LI wrote:
I am thinking to do re-fragment in netfront for these skbs like following,

Create a new skb, copy linear data and frag data from original skb into
this one, and make every frags data size is PAGE_SIZE except for the
last fragment. It is possible that the last fragment length is less than
PAGE_SIZE, then free the original skb. The skb packet is large, and
there will be lots of copys.
You don't need (or I suspect want) to copy, you can directly add pages
from the source skb's frags to the destination skb's frags, with
appropriate refcount frobbing. You can also share a page between two (or
more) skbs in the case where the boundary between two skbs happens to be
in the middle of a page.

But more importantly than all that you need to do more than just
refragment, you actually need to resegment i.e. you need to duplicate
the headers (Ethernet, IP, TCP) at the front of each new skb and adjust
the (psuedo-)checksums as appropriate (which will depend on whether the
SKB is GSO, or just has checksum offload or nothing).

So you need to go from
   <Ether><IP><TCP><...data... ...data...>
   <Ether><IP><TCP><...data...>  <Ether><IP><TCP><...data...>

Where the headers are adjusted to cope with this. Some of data might be
in frags but equally some might be in skb->data.

I'm guessing that Linux already has code which can do this for you,
since it has a software fallback for GSO.

There is "skb_gso_segment" existing to perform segmentation on skb and return a list of segments. From the code, it seems this function is specific to GSO packets, maybe it is not available to other non-gso large packets which requires slots is larger than MAX_SKB_FRAGS + 1". So for non-gso offload packets, I would go forward to write a function to do resegment.



struct skbuff *xennet_refrag_skb(skb)
     create newskb
     copying data and doing fragmentation
     return newskb


if (unlikely(slots>   MAX_SKB_FRAGS + 1)) {
                "xennet: skb rides the rocket: %d slots\n", slots);
          skb = xennet_refrag_skb(skb);

"netchannel vs MAX_SKB_FRAGS". Maybe these two mechanism are all
Lets see first if this is indeed the problem. Perhaps a simple debug
patch that just does:

        #define DEBUG_MAX_FRAGS 21

in both netback and netfront to set the maximum number of frags we can
handle to 21? If that works with Sander test - then yes, it looks like
we really need to get this 'feature-max-skb-frags' done.

Xen-devel mailing list

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.