Re: [Xen-devel] Question about TCP checksum offload in Xen

[John Haxby <john.haxby@xxxxxxxxxx>]

On 05/12/13 11:39, Ian Campbell wrote:
> On Thu, 2013-12-05 at 11:29 +0000, Anil Madhavapeddy wrote:
>> > On Tue, Dec 03, 2013 at 01:00:23PM +0000, Balraj Singh wrote:
>>> > > Hi,
>>> > > 
>>> > > I'm working on verifying TCP checksums on incoming packets in Mirage, 
>>> > > but
>>> > > I've run into a bit of a problem.
>>> > > 
>>> > > If TCP checksum offload is turned on on a virtual interface (this is the
>>> > > default), and if the TCP connection is local to the machine, it looks 
>>> > > like
>>> > > Xen does not calculate the checksum at all.  This may be valid because 
>>> > > Xen
>>> > > may be providing a stronger guarantee, but it means that incoming 
>>> > > packets
>>> > > don't have a valid checksum in the header.  This then means that in 
>>> > > Mirage
>>> > > we can't just have checksum verification turned on all the time.  This
>>> > > would have been the safe fall back option and detecting that checksum
>>> > > offload is on, and then not duplicating the verification in Mirage would
>>> > > have been an optimisation.  But it looks like this is not an option.  
>>> > > Now I
>>> > > need to know for every incoming packet whether checksum verification 
>>> > > should
>>> > > be done or not.  It should ideally be for every packet since chksum 
>>> > > offload
>>> > > can be turned off and on on the VIF and existing tcp connections should
>>> > > continue.  If not every packet, I need to get a notification or 
>>> > > efficiently
>>> > > detect right away that the setting is changed on the VIF.
>> > 
>> > This is a question that seems to keep coming up even for Linux and
>> > Windows, as the combination of local<->local VMs vs local<->off-host and
>> > the checksum offload is quite confusing.
>> > 
>> > CCing xen-devel: is the appropriate behaviour for a guest VM that wants to
>> > use checksum offloading in all situations documented anywhere?
> I don't understand the question/concern. If you have enabled checksum
> offload then of course you don't recalculate the checksum, that's the
> whole point of offloading it.

I get this a lot.

There are a few different cases:

  * domain to domain traffic
  * domain to external traffic with egress from a NIC that does offload
  * domain to external through a non-offloading NIC

With xen checksum offloading, domain to domain traffic appears to be
received with a bad checksum.  This is OK, there is no point in
calculating a checksum if the packets are only going through memory.  If
your memory is going to randomly corrupt packets you have more bigger
problems to worry about.   However, this does upset at least Solaris: if
you're using a Solaris guest for NAT then the NAT module on Solaris gets
all upset if the checksum is wrong and drops the packets.  (This is
Solaris's NAT module being overly picky, it may need to recalculate or
at least invalidate the existing checksum, but it doesn't need to check
it as well.)

The second two cases are of interest from the domain perspective.  A
domain has no way of knowing how any given packet is going to leave the
host (or even if it is) so it can't know ahead of time whether to
calculate any checksums: the skb's are just marked with "checksum
needed" as usual and either the egress NIC will do the job or dom0 will
do it.

There is absolutely nothing wrong in any of this (Solaris
notwithstanding).   The difficulty is getting people to realise that
checksums are only calculated when a packet hits the cat-5.  It doesn't
need documenting, it just needs a little thought.   I got tired of
hammering the point home :)

jch

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel