Re: [Xen-devel] Poor network performance between DomU with multiqueue support

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Fri, Dec 05, 2014 at 01:17:16AM +0000, Zhangleiqiang (Trump) wrote:
[...]
> > I think that's expected, because guest RX data path still uses grant_copy 
> > while
> > guest TX uses grant_map to do zero-copy transmit.
> 
> As far as I know, there are three main grant-related operations used in split 
> device model: grant mapping, grant transfer and grant copy. 
> Grant transfer has not used now, and grant mapping and grant transfer both 
> involve "TLB" refresh work for hypervisor, am I right?  Or only grant 
> transfer has this overhead?

Transfer is not used so I can't tell. Grant unmap causes TLB flush.

I saw in an email the other day XenServer folks has some planned
improvement to avoid TLB flush in Xen to upstream in 4.6 window. I can't
speak for sure it will get upstreamed as I don't work on that.

> Does grant copy surely has more overhead than grant mapping? 
> 

At the very least the zero-copy TX path is faster than previous copying
path.

But speaking of the micro operation I'm not sure.

There was once persistent map prototype netback / netfront that
establishes a memory pool between FE and BE then use memcpy to copy
data. Unfortunately that prototype was not done right so the result was
not good.

> >From the code, I see that in TX, netback will do gnttab_batch_copy as well 
> >as gnttab_map_refs:
> 
> <code> //netback.c:xenvif_tx_action
>       xenvif_tx_build_gops(queue, budget, &nr_cops, &nr_mops);
> 
>       if (nr_cops == 0)
>               return 0;
> 
>       gnttab_batch_copy(queue->tx_copy_ops, nr_cops);
>       if (nr_mops != 0) {
>               ret = gnttab_map_refs(queue->tx_map_ops,
>                                     NULL,
>                                     queue->pages_to_map,
>                                     nr_mops);
>               BUG_ON(ret);
>       }
> </code>
> 

The copy is for the packet header. Mapping is for packet data.

We need to copy header from guest so that it doesn't change under
netback's feet.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel