Re: [Xen-devel] [linux-4.1 test] 63030: regressions - FAIL

[Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>]

Ian Campbell writes ("Re: [Xen-devel] [linux-4.1 test] 63030: regressions - 
FAIL"):
> On Thu, 2015-10-22 at 12:03 +0100, Wei Liu wrote:
> > No, vif-bridge script has two runes for off-lining a vif
> >   brctl delif $bridge $vif
> >   ifconfig $vif down
> > 
> > Neither of these causes cache entry to be flushed.
> 
> $vif disappearing when netback finally deletes the device will though. Or
> it should/used to.
> 
> Maybe this is happening after the new guest has started and confusing
> things somewhere?


There is confusion here.  Someone used the phrase `arp cache'.  But
there are actually two relevant runtime of MAC addresses:

 * Each host has a neighbour database mapping IPv4 addresses to MAC
   addresses.  This is used when trying to pass on an IPv4 datagram to
   a host on the same ethernet (same broadcast domain).  This database
   is normally referred to as an `ARP cache'.  Addresses are added to
   the table by both ARP requests and responses, and also in many
   implementations entries are refreshed by ordinary traffic.

   In the test colo, the osstest VM is on the same (bridged) ethernet
   as the test box so, the relevant arp cache is the one in the
   osstest controller's kernel: the osstest controller wants to send
   an ssh SYN to the guest, and needs to construct an ethernet frame
   with the guest's MAC address.  This is done using the osstest
   controller's ARP cache entry.

   The osstest controller's ARP cache is unaffected by the migration.
   ARP cache entries do time out but only after a number of minutes,
   and the guest will have been spoken to recently by the controller.
   I have no reason to think that lack of an entry for the guest's
   IPv4 address in the osstest controller's ARP cache is relevant.

 * Each bridge has a forwarding database mapping MAC addresses to its
   outbound links.  This is normally referred to as the bridge
   (switch) `learning', and the table as the `MAC address table'.  MAC
   addresses are learned when switch sees incoming frames.  When the
   bridge receives a frame for a destination MAC address for which it
   has no entry, it forwards the frame out of all its ports.  Special
   considerations apply to broadcast and multicast MAC addresses.
   None of this involves IPv4 or IPv6 addresses.

   In the test colo in the migration test case, there are up to four
   relevant bridges:

      * The source test box's dom0's software bridge.
        This has (logically speaking) three `ports':
          - the test box's physical network interface
          - the dom0 itself
          - the vif corresponding to the outbound guest
          - in a single-host test, the vif corresponding to
            the inbound guest

      * The physical switch connecting the test boxes and the VM host
        (newcastle.test-lab.xenproject.org).  This has two or three
        relevant physical ports, for the two or three relevant
        physical machines.  (In fact there are VLANs involved but this
        is not relevant.)

      * The software bridge in newcastle.  This has two relevant
        ports:
          - newcastle's physical interface
          - the vif serving the osstest VM
        
      * In a two-machine test, rather than a localhost test, the
        destination test box's dom0's software bridge, which parallels
        the source test box's.

   When the guest stops running on the source (with its vif torn
   down), and starts running on the destination:

     (a) The source test box software bridge should lose its MAC
        address table entry for the guest, because the corresponding
        port (the vif) is removed.  However I am not sure whether this
        actually happens immediately in Linux.

        It may be that instead the MAC address table entry for the
        guest remains present but points to the dead vif.  In this
        case incoming frames from the wire, the for the guest will be
        dropped.

     (b) The destination test box (if different) will come up without a
        MAC address entry for the guest.  If a frame for the guest's
        MAC address arrives at the physical interface, it will be
        forwarded to all of the other interfaces enslaved to the
        bridge: ie, to the dom0 (which will ignore it because it has
        the wrong destination MAC address) and to the newly-created
        guest.

     (c) In a two-host test, the physical switch connecting the two
        test boxes will retain the wrong learnt switch port.  It will
        forward frames for the guest (only) to the source test box,
        rather than the destination test box, where they will be
        discarded.

   It is (a) and (c) that the gratuitous ARP is supposed to fix.

   The guest is supposed to send, when its interface comes up after
   migration, a single broadcast gratuitous ARP response containing
   its own IPv4 and MAC addresses.

   The IPv4 address in this message is irrelevant.

   The purpose is to update the MAC address tables in all the switches
   in the network.  Each switch which receives the gratuitous ARP
   updates its MAC address table to map the guest's MAC address to the
   port on which the gratuitous ARP was recevied.

   If this happens, then frames from everywhere on the ethernet, to
   the guest, will be properly delivered.  If it doesn't then there
   may be lost packets and/or low-level timeouts of various kinds.


Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel