RE: [Xen-devel] RFC: [0/2] Remove netloop by lazy copying in netback

["Santos, Jose Renato G" <joserenato.santos@xxxxxx>]

> -----Original Message-----
> From: Ian Pratt [mailto:Ian.Pratt@xxxxxxxxxxxx] 
> Sent: Friday, March 23, 2007 6:24 AM
> To: Herbert Xu; Keir Fraser
> Cc: Xen Development Mailing List; Santos, Jose Renato G; 
> ian.pratt@xxxxxxxxxxxx
> Subject: RE: [Xen-devel] RFC: [0/2] Remove netloop by lazy 
> copying in netback
> 
> > The only catch is that I wanted to have the same lazy-copy 
> mechanism 
> > for dom0=>domU.  In that case we can't rely on domU to trigger the 
> > copy so dom0 (or the driver domain) will need to initiate it.  The 
> > easiest way to do that from outside domU is through 
> changing its P2M 
> > table.
> > 
> > I suppose doing it this way doesn't necessarily preclude a 
> different 
> > solution for dom0=>domU.
> 
> I think we can make the guest TX path work rather better by 
> putting the tx packet header off into a separate fragment and 
> placing it within one of a set of pages that are 
> 'permanently' mapped between the front and back ends. The 
> backend would not typically grant map the payload 
> fragment(s), but only do so if it catches a page fault trying 
> to access them.  
> 
> The way I'd do the 'permanent' mappings is to have a bit in 
> the grant reference passed across the ring that indicates 
> that the backend is free to leave the grant mapped. The bit 
> should be set whenever the grant entry is used (the front end 
> can't mix and match). The backend thus just needs to keep 
> track of which grant refs it has mapped via this permanent 
> scheme so that it can spot (valid) reuse, and clean up 
> afterwards. Since the front end will typically use a block of 
> grant refs for this purpose the backend code can be optimised 
> for a simple range check. The frontend is free to pack 
> multiple header fragments on to a page to save memory (rather 
> than using larger payload sized frags).
> 
> Jose Renato investigated something similar to this last year, 
> but AFAIK we didn't take it the whole way of introducing 
> permanent mappings with no (default) mapping of payloads. 
> 
>

  The way I did it was slightly different.
  The shared pages used to store the packet headers are  
  mapped at driver initialization at the same time the
  I/O ring is initialized (You can think of these pages
  as an extension to the I/O ring and mapped and unmapped at
  the same time). This shared header area
  is divided into fixed size slots large enough to store all
  packet headers (MAC/IP/TCP). Netfront copies the
  header of the packet to a free slot on the shared header
  buffer area and passes the offset of the header on the
  first I/O ring request for the packet (instead of
  passing a grant reference). Payload data are transferred
  as additional requests on I/O ring using grant
  references as usual. Netback assumes the first request
  of all packets point to headers in the shared area and
  copies the header into the main socket buffer data area.
  Payload data (in additional I/O ring requests) are associated
  with skb fragments but not mapped into dom0 memory 
  unless a page fault happens
  (indicating that dom0 is trying to access that data).
  If the packet is sent out on the wire, the payload is never  
  touched by the CPU (assuming no netfilter rules, etc...)
  and no memory mapping is necessary.

  I have put this code on the shelf for a while as I have been
  busy on analyzing/optimizing the receive path.
  I plan to port my patch to the current xen-unstable
  and get some performance data for the Xen summit after
  I finish my experiments on the receive side.
  I think we should include this on future releases, but
  I would like to hold on until I have some other results
  that may influence how we do this.
  This would be a good topic for discussion on the summit

  Regards
  
  Renato

 
> BTW: when adding fragment support you 'borrowed' the 
> descriptor id field which wasn't being used at the time.  
> This means we can no longer return free buffers to the rx 
> ring out of order. This is rather unfortunate as its very 
> useful for dealing with some kinds of smart NIC. I expect 
> we'll have to introduce a net ring v2 format in the not too 
> distant future unless we can think of a cunning way of 
> stealing them back?
> 
> Thanks,
> Ian
> 
> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel