[Xen-users] Trouble with Xen networking

["Erik G. Burrows" <erik@xxxxxxxxxxxxxxx>]

All,
I'd appreciate help diagnosing the following problem I'm having getting
networking under Xen working.

First, I have this exact same setup working on other types of hosts, so
something specific to this type of host, Appro S1321, is causing the
problem. Also, I have tried it on three different hosts of this same
model, with the exact same results.

So, here's the setup:
  Appro S1321 host
  32GB RAM
  2x AMD Opteron 285
  2x Broadcom BCM5721 NICs (tg3 driver in dom0)
  OS: OVM Server v. 2.2.1
  Xen 3.4.0

When booting the system clean, the interfaces and routes look like this:
eth0      Link encap:Ethernet  HWaddr 00:D0:68:10:00:D1
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:248 errors:0 dropped:0 overruns:0 frame:0
          TX packets:247 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:41596 (40.6 KiB)  TX bytes:32460 (31.6 KiB)
          Interrupt:21

eth1      Link encap:Ethernet  HWaddr 00:D0:68:10:00:D0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:22

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:166 errors:0 dropped:0 overruns:0 frame:0
          TX packets:166 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:20533 (20.0 KiB)  TX bytes:20533 (20.0 KiB)

xenbr0    Link encap:Ethernet  HWaddr 00:D0:68:10:00:D1
          inet addr:10.12.12.110  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:315 errors:0 dropped:0 overruns:0 frame:0
          TX packets:317 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:52156 (50.9 KiB)  TX bytes:39120 (38.2 KiB)

xenbr1    Link encap:Ethernet  HWaddr 00:D0:68:10:00:D0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)


Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.12.12.0      0.0.0.0         255.255.255.0   U     0      0        0
xenbr0
0.0.0.0         10.12.12.1      0.0.0.0         UG    0      0        0
xenbr0


When I start a guest, it creates vif1.0, as:

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:1156 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

And adds the vif to the bridge:

bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.00d0681000d1       no              vif1.0
                                                        eth0
xenbr1          8000.00d0681000d0       no              eth1

The problem, is that the dom0 OS can ping/otherwise-connect-to the domU,
and visa-versa, but the domU cannot get to the outside world.

If I try to ping the domU from another host on the same subnet (on a dumb
switch, nothing fancy), I get no response at all.

Tcpdumping eth0/xenbr0 on dom0 doesn't even show the icmp requests coming
in. I have run "ifconfig eth0 promisc", and this does not help.

On a fancier switch, I spanned the system's network port to another host,
and ran tcpdump on that. From there, I can see the ICMP requests coming
in, so I know it's not some MAC-table mix-up on the switch.

If, before starting the guest VM, I run "/etc/init.d/network restart",
then start the guest, networking will actually work for it, for a short
time (~1 min). All I can see that this accomplishes is shutting down the
eth1 nic, and assigning the xenbr0 IP address to eth0. If I do this
manually, nothing useful happens.

Any ideas? As I said before, I have this exact same software setup working
on other machine types. Even the same VM image will work on another
machine...

Thanks,
  Erik Burrows



-- 
If you are flammable and have legs, you're never blocking a fire exit.
-Mitch Hedberg


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users