[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Xen-users] help--dom0 network goes unpingable when xend starts
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
timm@xxxxxxxx http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.
On Mon, 28 Apr 2008, Todd Deshane wrote:
To get rid of the xenbr1 manually, you should be able to do:
brctl delif <any interface that is on that bridge>
then, when no more interfaces do:
ifconfig xenbr1 down
brctl delbr xenbr1
This worked, xenbr1 is gone.
Do you get any warning or error to stdout or to the /var/log/xen/* log
files when starting/restarting xend?
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VBD.set_device
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VBD.set_mode
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VBD.set_type
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
VM.get_auto_power_on not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
VM.set_auto_power_on
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
VM.set_VCPUs_max not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
VM.set_VCPUs_at_startup not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: debug.get_all
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
console.get_other_config not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
console.set_other_config not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VIF.get_network
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VIF.set_device
not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VIF.set_MAC not
found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: VIF.set_MTU not
found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
session.get_all_records not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call:
event.get_record not found
[2008-04-17 15:57:00 3711] WARNING (XendAPI:672) API call: event.get_all
not found
Have you tried to start the network-bridge script on the command line
manually with:
/etc/xen/scripts/network-bridge start
I tried this by hand.
/etc/xen/scripts/network-bridge start
Nothing to flush.
Nothing to flush.
Waiting for peth0 to negotiate link....
...and then it hangs forever.
Machine then goes unpingable.
Restarting network doesn't help.
rebooting, but not restarting xen, we get the network back.
It is also not clear to me anymore what works and what doesn't. Could
you clarify that again?
Thanks,
Todd
Basic problem: dom0 can't ping the outside world or be pinged
by the outside world.
domU can ping dom0 and the outside world.
dom0 can ping domU but nothing else.
Steve timm
On Mon, Apr 28, 2008 at 11:39 AM, Steven Timm <timm@xxxxxxxx> wrote:
Here's the output of
[root@fnpcsrv5 ~]# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif2.0
vif1.0
peth0
vif0.0
xenbr1 8000.000000000000 no
and here's xend-config.sxp
[root@fnpcsrv5 xen]# grep net xend-config.sxp | grep -v ^#
(network-script network-bridge)
And here's what the domU config files show at the moment:
vif = [ 'mac=00:16:3e:0f:05:01, bridge=xenbr0' ]
------------
I was at one point running a second bridge, but took it out
during the debugging. Nevertheless the xenbr1 doesn't go away.
and I haven't figured out any brctl commands that will make it go away.
From a similarly-configured working machine on different hardware, this
is what I wanted to do (and what I had originally configured on this one
too), and has been working fine for me on many other
machines.
[root@fermigrid5 ~]# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif12.0
vif10.0
vif9.0
vif8.0
vif7.0
peth0
vif0.0
xenbr1 8000.feffffffffff no vif12.1
vif10.1
vif9.1
vif8.1
vif7.1
peth1
vif0.
-------------------------------
[root@fermigrid5 xen]# grep net xend-config.sxp | grep -v ^#
(network-script my-network-bridge)
[root@fermigrid5 xen]# more scripts/my-network-bridge
#!/bin/sh
# start bridges on both eth0 and eth1
XENDIR="/etc/xen/scripts"
$XENDIR/network-bridge "$@" netdev=eth0 bridge=xenbr0 vifnum=0
$XENDIR/network-bridge "$@" netdev=eth1 bridge=xenbr1 vifnum=1
[root@fermigrid5 xen]#
vif = [ 'mac=00:16:3e:05:05:01, bridge=xenbr0', 'mac=00:16:3e:05:05:0a,
bridge=xenbr1' ]
Hope this helps us figure out what is wrong.
Have seen a couple places on the net that a bios upgrade
and-or tweak to the broadcom controller is needed
for the PE1950 to fix related problems. Has anyone tried?
Steve Timm
On Mon, 21 Apr 2008, Todd Deshane wrote:
On Thu, Apr 17, 2008 at 9:24 PM, Steven Timm <timm@xxxxxxxx> wrote:
I installed 64-bit xen 3.1.0 (from xensource.com tarballs) on
three new machines today, using a configuration setup that I've
used successfully many times before. However, I encountered a
new problem.
These are Dell Poweredge 1950 servers, by the way.
From lspci
04:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708
Gigabit Ethernet (rev 12)
08:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708
Gigabit Ethernet (rev 12)
from lspci
Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.4.44 (August 10,
2006)
ACPI: PCI Interrupt 0000:08:00.0[A] -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz
found at
mem f4000000, IRQ 16, node addr 0019b9ec40ba
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz
found at
mem f8000000, IRQ 16, node addr 0019b9ec40b8
---------------
note that the Xen kernel 2.6.18 picks the opposite mac addresses as eth0
from what the redhatized non-xen kernel does. This is
undone by ifcfg-eth0.
When the xen kernel boots, before xend starts, I can see the outside
network just fine.
[root@fnpcsrv3 xen]# netstat -nNr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt
Iface
192.168.167.0 0.0.0.0 255.255.255.0 U 0 0 0
eth1
131.225.166.0 0.0.0.0 255.255.254.0 U 0 0 0
eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0
eth1
0.0.0.0 131.225.167.200 0.0.0.0 UG 0 0 0
eth0
[root@fnpcsrv3 xen]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:19:B9:EC:40:B8
inet addr:131.225.166.97 Bcast:131.225.167.255 Mask:
255.255.254.0
inet6 addr: fe80::219:b9ff:feec:40b8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:591697 errors:0 dropped:0 overruns:0 frame:0
TX packets:3060 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:38067586 (36.3 MiB) TX bytes:395536 (386.2 KiB)
Interrupt:16 Memory:f8000000-f8011100
eth1 Link encap:Ethernet HWaddr 00:19:B9:EC:40:BA
inet addr:192.168.167.3 Bcast:192.168.167.255
Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:16 Memory:f4000000-f4011100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:192 errors:0 dropped:0 overruns:0 frame:0
TX packets:192 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:15314 (14.9 KiB) TX bytes:15314 (14.9 KiB)
--------------------------------------------------------------
Now here's ifconfig from an identical system once xend is turned on
[root@fnpcsrv5 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:19:B9:EC:4A:21
inet addr:131.225.166.100 Bcast:131.225.167.255 Mask:
255.255.254.0
inet6 addr: fe80::219:b9ff:feec:4a21/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:508292 errors:0 dropped:0 overruns:0 frame:0
TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:30786266 (29.3 MiB) TX bytes:1658 (1.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:21 errors:0 dropped:0 overruns:0 frame:0
TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1916 (1.8 KiB) TX bytes:1916 (1.8 KiB)
peth0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:523679 errors:0 dropped:0 overruns:0 frame:0
TX packets:15964 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:33836052 (32.2 MiB) TX bytes:1132609 (1.0 MiB)
Interrupt:16 Memory:f4000000-f4011100
vif0.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:33 errors:0 dropped:0 overruns:0 frame:0
TX packets:508293 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1658 (1.6 KiB) TX bytes:30786336 (29.3 MiB)
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:7848 errors:0 dropped:0 overruns:0 frame:0
TX packets:499340 errors:0 dropped:159 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:347417 (339.2 KiB) TX bytes:30239848 (28.8 MiB)
vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:7867 errors:0 dropped:0 overruns:0 frame:0
TX packets:496186 errors:0 dropped:191 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:346478 (338.3 KiB) TX bytes:30050363 (28.6 MiB)
xenbr0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:508099 errors:0 dropped:0 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:23650570 (22.5 MiB) TX bytes:90 (90.0 b)
xenbr1 Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:468 (468.0 b)
------------------------------------------------------------------------
As part of the debugging I dialed back my configuration, which normally
has a xenbr0 and a xenbr1, to use a just a xenbr0 and have just one
network interface on each domU and on the dom0. Nevertheless the
problem is the same and I can't seem to get rid of the xenbr1.
I note that at the start of the xend there is a timeout of about 10-15
seconds as it is trying to turn on the second bridge.
Now here is the really strange part. While logged in on the
console of the dom0, I can go ahead and start xen domU's, and they
go ahead and boot up normally and can see the outside network fine.
[root@fnpcsrv5 ~]# xm list
Name ID Mem VCPUs State
Time(s)
Domain-0 0 1953 8 r-----
127.3
fnpc5x1 1 6000 4 -b----
23.1
fnpc5x4 2 2000 1 -b----
20.6
[root@fnpcsrv5 ~]#
Oh, and by the way, dom0 is pingable from the domU's although
it cannot be seen from the outside net.
What should I be looking at?
Could you also provide the output of:
brctl show
grep net /etc/xen/xend-config.sxp | grep -v '#'
Also, in the domU config files what are you specifying for the bridge?
Are you running any custom network scripts (these would be specified in
/etc/xen/xend-config.sxp and should have shown up in the output of the
grep
above.
Cheers,
Todd
Steve Timm
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
timm@xxxxxxxx http://home.fnal.gov/~timm/
<http://home.fnal.gov/%7Etimm/>
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group
Leader.
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
--
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
timm@xxxxxxxx http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group
Leader.
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
