[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-users] xen hotplug scripts not working
Hello Everyone, I have set up a xen server with 10 physical network devices using the bridge mode of XEN. This is because I want to split and separate the network traffic for security reasons. Maybe the best way is to use pci back hide. But I want to understand what is going on here. For all testings I have only activated eth[0-3]. My first tests were successful. I have created a VM using 'xm'. But we got some network problems: root@bs-xen-p-01:~[0]# netstat -rnv Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 10.1.17.10 10.1.1.1 255.255.255.255 UGH 0 0 0 eth0 10.1.17.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3 10.1.32.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 10.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 10.1.19.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 0.0.0.0 10.1.1.1 0.0.0.0 UG 0 0 0 eth0 root@bs-xen-p-01:~[0]# The routing for eth3, eth3 and eth1 points back to his source. To get partial rid of the network problems we first define the first rule (we administrate the xen server through a portal running in the 10.1.17.0/24 segment). Then I have had the idea to define the physical interfaces in network config to some IP's residing in the same subnet as the xen server itself (10.1.1.0/24) i. e.: auto eth1 allow-hotplug eth1 iface eth1 inet static address 10.1.1.201 netmask 255.255.255.0 network 1.1.1.0 But since I have done this and restarted the network, the VM cannot start any more. I have rebooted the machine - nothing I have set back the network interface - nothing I have rebooted the machine - nothing ifconfig shows only eth[0-9] and lo. bridge control: root@bs-xen-p-01:~[0]# brctl show bridge name bridge id STP enabled interfaces root@bs-xen-p-01:~[0]# All I've got is a message: root@bs-xen-p-01:~[0]# xm create otrs-test.sxp Using config file "/etc/xen/otrs-test.sxp". Error: Device 0 (vif) could not be connected. Hotplug scripts not working. root@bs-xen-p-01:~[0]# The orts-test.sxp file: name = "otrs-test" kernel = "/boot/vmlinuz-2.6.18-4-xen-amd64" ramdisk = "/boot/initrd.img-2.6.18-4-xen-amd64" memory = 512 disk = [ 'phy:lvm-vg1/vm-otrs-test,sda1,w' ] root = "/dev/sda1 ro" vif = [ 'ip=10.1.19.20,mac=00:16:de:ad:fa:cd,bridge=xenbr1' ] dhcp="off" netmask="255.255.255.0" gateway="10.1.19.1" hostname="otrs-test" If I set vif to [ '' ] and try starting the VM: root@bs-xen-p-01:~[0]# xm create otrs-test.sxp Using config file "/etc/xen/otrs-test.sxp". Error: Device 0 (vif) could not be connected. Backend device not found. root@bs-xen-p-01:~[1]# So I start the bridge manually: /etc/xen/scripts/network-bridge start vifnum=0 bridge=xenbr0 netdev=eth0 /etc/xen/scripts/network-bridge start vifnum=1 bridge=xenbr1 netdev=eth1 /etc/xen/scripts/network-bridge start vifnum=2 bridge=xenbr2 netdev=eth2 root@bs-xen-p-01:~[0]# dmesg|grep xenbr0 xenbr0: port 1(vif0.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 1(vif0.0) entering forwarding state xenbr0: port 2(peth0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 2(peth0) entering forwarding state xenbr0: port 1(vif0.0) entering disabled state xenbr0: port 2(peth0) entering disabled state xenbr0: port 2(peth0) entering disabled state xenbr0: port 1(vif0.0) entering disabled state root@bs-xen-p-01:~[0]# dmesg|grep xenbr1 xenbr1: port 1(vif0.1) entering learning state xenbr1: topology change detected, propagating xenbr1: port 1(vif0.1) entering forwarding state xenbr1: port 2(peth1) entering learning state xenbr1: topology change detected, propagating xenbr1: port 2(peth1) entering forwarding state root@bs-xen-p-01:~[0]# dmesg|grep xenbr2 xenbr2: port 1(vif0.2) entering learning state xenbr2: topology change detected, propagating xenbr2: port 1(vif0.2) entering forwarding state xenbr2: port 2(peth2) entering learning state xenbr2: topology change detected, propagating xenbr2: port 2(peth2) entering forwarding state root@bs-xen-p-01:~[0]# ifconfig shows eth[0-3], lo, peth[1-2], vif0.[1-2] and xenbr[1-2]. All interfaces up and running and have a link status. I fired up the VM, the VM starts, but ends up in the maintenance mode because the filesystem of the VM is mounted read only. If I gave the root password to enter the maintenance mode I can see the networking is not startet (ifconfig shows nothing). On the xen side: root@bs-xen-p-01:~[0]# brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.feffffffffff no vif0.0 peth0 xenbr1 8000.feffffffffff no vif0.1 peth1 vif6.0 xenbr2 8000.feffffffffff no vif0.2 peth2 root@bs-xen-p-01:~[0]# I have viewed /var/log/xen/*, /var/log/messages and dmesg. The only hint I have found is in xen-hotplug.log: Nothing to flush. bridge xenbr1 does not exist! So I switched on (set -x) the "debug" mode in the xen-hotplug-common.sh. Then I rebooted the machine. After rebooting I only try to start the VM. You can see the result in the attached file. I have seen several 'trap sigerr ERR' but I don't understand where they come from. Could someone help me, not only explaining the last error. Are there any suggestions setting up a XEN server with many physical network interfaces? Christian Placzek Nothing to flush. bridge xenbr1 does not exist! Nothing to flush. bridge xenbr1 does not exist! Nothing to flush. bridge xenbr1 does not exist! Nothing to flush. bridge xenbr1 does not exist! +++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ export LANG=POSIX +++ LANG=POSIX ++++ set ++++ grep '^LC_' ++++ cut -d= -f1 +++ unset +++ trap sigerr ERR +++ log debug add XENBUS_PATH=backend/vbd/5/2049 +++ local level=debug +++ shift +++ logger -p daemon.debug -- /etc/xen/scripts/block: add XENBUS_PATH=backend/vbd/5/2049 ++ findCommand add ++ for arg in '"$@"' ++ expr index add = +++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ export LANG=POSIX +++ LANG=POSIX ++++ set ++++ grep '^LC_' ++++ cut -d= -f1 ++ command=add ++ return ++ '[' add '!=' add ']' ++ XENBUS_PATH=backend/vbd/5/2049 ++ xenstore_read_default backend/vbd/5/2049/type MISSING ++ xenstore-read backend/vbd/5/2049/type +++ unset +++ trap sigerr ERR +++ log debug online XENBUS_PATH=backend/vif/5/0 +++ local level=debug +++ shift +++ logger -p daemon.debug -- /etc/xen/scripts/vif-bridge: online XENBUS_PATH=backend/vif/5/0 ++ . /etc/xen/scripts/xen-network-common.sh +++ '[' -e /etc/SuSE-release ']' +++ which ifup ++ findCommand online ++ for arg in '"$@"' ++ expr index online = ++ command=online ++ return ++ '[' online '!=' online ']' ++ case "$command" in ++ evalVariables online ++ for arg in '"$@"' ++ expr index online = '>' 1 ++ ip= +++ xenstore_read_default backend/vif/5/0/ip '' +++ xenstore-read backend/vif/5/0/ip + t=phy + case "$command" in ++ xenstore_read_default backend/vbd/5/2049/physical-device MISSING ++ xenstore-read backend/vbd/5/2049/physical-device ++ ip=10.1.19.20 ++ XENBUS_PATH=backend/vif/5/0 ++ vif=vif5.0 +++ xenstore_read_default backend/vif/5/0/vifname '' +++ xenstore-read backend/vif/5/0/vifname ++ echo MISSING + phys=MISSING + '[' MISSING '!=' MISSING ']' + '[' -n phy ']' ++ xenstore_read backend/vbd/5/2049/params +++ xenstore-read backend/vbd/5/2049/params +++ echo '' ++ vifname= ++ '[' '' ']' + bridge= ++ xenstore_read_default backend/vif/5/0/bridge '' ++ xenstore-read backend/vif/5/0/bridge ++ local v=lvm-vg1/vm-otrs-test ++ '[' lvm-vg1/vm-otrs-test '!=' '' ']' ++ echo lvm-vg1/vm-otrs-test + p=lvm-vg1/vm-otrs-test ++ xenstore_read backend/vbd/5/2049/mode +++ xenstore-read backend/vbd/5/2049/mode + bridge=xenbr1 + '[' -z xenbr1 ']' + case "$command" in + setup_bridge_port vif5.0 + local dev=vif5.0 + ip link set vif5.0 down + ip link set vif5.0 arp off + ip link set vif5.0 multicast off ++ local v=w ++ '[' w '!=' '' ']' ++ echo w + ip link set vif5.0 addr fe:ff:ff:ff:ff:ff + mode=w + case $t in ++ expand_dev lvm-vg1/vm-otrs-test ++ local dev ++ case $1 in ++ dev=/dev/lvm-vg1/vm-otrs-test ++ echo -n /dev/lvm-vg1/vm-otrs-test + dev=/dev/lvm-vg1/vm-otrs-test ++ xenstore_read backend/vbd/5/2049/frontend-id +++ xenstore-read backend/vbd/5/2049/frontend-id + ip addr flush vif5.0 Nothing to flush. + add_to_bridge xenbr1 vif5.0 + local bridge=xenbr1 + local dev=vif5.0 + '[' -e /sys/class/net/xenbr1/brif/vif5.0 ']' + brctl addif xenbr1 vif5.0 ++ local v=5 ++ '[' 5 '!=' '' ']' ++ echo 5 + FRONTEND_ID=5 ++ xenstore_read_default /local/domain/5/vm unknown ++ xenstore-read /local/domain/5/vm bridge xenbr1 does not exist! + FRONTEND_UUID=/vm/4e2612fa-d832-b56e-661a-833fa4b8aa72 + claim_lock block + local lockdir=/var/run/xen-hotplug/block + mkdir -p /var/run/xen-hotplug + _claim_lock /var/run/xen-hotplug/block + local lockdir=/var/run/xen-hotplug/block ++ _lock_owner /var/run/xen-hotplug/block ++ cat /var/run/xen-hotplug/block/owner ++ echo unknown + local owner=unknown + local retries=0 + '[' 0 -lt 100 ']' + mkdir /var/run/xen-hotplug/block + trap 'release_lock /var/run/xen-hotplug/block; sigerr' ERR + _update_lock_info /var/run/xen-hotplug/block + echo '12596: /etc/xen/scripts/block' + return + check_device_sharing /dev/lvm-vg1/vm-otrs-test w + local dev=/dev/lvm-vg1/vm-otrs-test ++ canonicalise_mode w ++ local mode=w ++ expr index w w ++ expr index w '!' ++ echo w + local mode=w + local result + '[' w == '!' ']' ++ check_sharing /dev/lvm-vg1/vm-otrs-test w ++ local dev=/dev/lvm-vg1/vm-otrs-test ++ local mode=w +++ device_major_minor /dev/lvm-vg1/vm-otrs-test +++ stat -L -c %t:%T /dev/lvm-vg1/vm-otrs-test ++ local devmm=fe:1 ++ local file ++ '[' w == w ']' ++ toskip='^$' +++ cat /proc/mounts +++ grep -v '^$' +++ cut -f 1 -d ' ' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e rootfs ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e none ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e none ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e udev ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e /dev/sda5 ']' +++ device_major_minor /dev/sda5 +++ stat -L -c %t:%T /dev/sda5 ++ local d=8:5 ++ '[' 8:5 == fe:1 ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e /dev/sda5 ']' +++ device_major_minor /dev/sda5 +++ stat -L -c %t:%T /dev/sda5 ++ local d=8:5 ++ '[' 8:5 == fe:1 ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e tmpfs ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e usbfs ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e tmpfs ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e devpts ']' ++ for file in '$(cat /proc/mounts | grep -v "$toskip" | cut -f 1 -d '\'' '\'')' ++ '[' -e /dev/sda2 ']' +++ device_major_minor /dev/sda2 +++ stat -L -c %t:%T /dev/sda2 ++ local d=8:2 ++ '[' 8:2 == fe:1 ']' ++ local base_path=backend/vbd +++ xenstore-list backend/vbd ++ for dom in '$(xenstore-list "$base_path")' +++ xenstore-list backend/vbd/5 ++ for dev in '$(xenstore-list "$base_path/$dom")' +++ xenstore_read_default backend/vbd/5/2049/physical-device '' +++ xenstore-read backend/vbd/5/2049/physical-device +++ echo '' ++ d= ++ '[' '' == fe:1 ']' ++ echo ok + result=ok + '[' ok '!=' ok ']' + write_dev /dev/lvm-vg1/vm-otrs-test + local mm ++ device_major_minor /dev/lvm-vg1/vm-otrs-test ++ stat -L -c %t:%T /dev/lvm-vg1/vm-otrs-test + mm=fe:1 + '[' -z fe:1 ']' + xenstore_write backend/vbd/5/2049/physical-device fe:1 + log debug 'Writing backend/vbd/5/2049/physical-device' 'fe:1 to xenstore.' + local level=debug + shift + logger -p daemon.debug -- /etc/xen/scripts/block: 'Writing backend/vbd/5/2049/physical-device' 'fe:1 to xenstore.' + xenstore-write backend/vbd/5/2049/physical-device fe:1 + success + xenstore_write backend/vbd/5/2049/hotplug-status connected + log debug 'Writing backend/vbd/5/2049/hotplug-status' 'connected to xenstore.' + local level=debug + shift + logger -p daemon.debug -- /etc/xen/scripts/block: 'Writing backend/vbd/5/2049/hotplug-status' 'connected to xenstore.' + xenstore-write backend/vbd/5/2049/hotplug-status connected + release_lock block + _release_lock /var/run/xen-hotplug/block + trap sigerr ERR + rm -rf /var/run/xen-hotplug/block + exit 0 +++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ export LANG=POSIX +++ LANG=POSIX ++++ grep '^LC_' ++++ cut -d= -f1 ++++ set +++ unset +++ trap sigerr ERR +++ log debug offline XENBUS_PATH=backend/vif/5/0 +++ local level=debug +++ shift +++ logger -p daemon.debug -- /etc/xen/scripts/vif-bridge: offline XENBUS_PATH=backend/vif/5/0 ++ . /etc/xen/scripts/xen-network-common.sh +++ '[' -e /etc/SuSE-release ']' +++ which ifup ++ findCommand offline ++ for arg in '"$@"' ++ expr index offline = ++ command=offline ++ return ++ '[' offline '!=' online ']' ++ '[' offline '!=' offline ']' ++ case "$command" in ++ evalVariables offline ++ for arg in '"$@"' ++ expr index offline = '>' 1 ++ ip= +++ xenstore_read_default backend/vif/5/0/ip '' +++ xenstore-read backend/vif/5/0/ip +++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +++ export LANG=POSIX +++ LANG=POSIX ++++ set ++++ grep '^LC_' ++++ cut -d= -f1 +++ unset +++ trap sigerr ERR +++ log debug remove XENBUS_PATH=backend/vbd/5/2049 +++ local level=debug +++ shift +++ logger -p daemon.debug -- /etc/xen/scripts/block: remove XENBUS_PATH=backend/vbd/5/2049 ++ findCommand remove ++ for arg in '"$@"' ++ expr index remove = ++ command=remove ++ return ++ '[' remove '!=' add ']' ++ '[' remove '!=' remove ']' ++ XENBUS_PATH=backend/vbd/5/2049 ++ xenstore_read_default backend/vbd/5/2049/type MISSING ++ xenstore-read backend/vbd/5/2049/type ++ ip=10.1.19.20 ++ XENBUS_PATH=backend/vif/5/0 ++ vif=vif5.0 +++ xenstore_read_default backend/vif/5/0/vifname '' +++ xenstore-read backend/vif/5/0/vifname + t=phy + case "$command" in + case $t in + exit 0 +++ echo '' ++ vifname= ++ '[' '' ']' + bridge= ++ xenstore_read_default backend/vif/5/0/bridge '' ++ xenstore-read backend/vif/5/0/bridge ++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ++ export LANG=POSIX ++ LANG=POSIX +++ set +++ grep '^LC_' +++ cut -d= -f1 ++ unset ++ trap sigerr ERR ++ log debug XENBUS_PATH=backend/vbd/5/2049 ++ local level=debug ++ shift ++ logger -p daemon.debug -- /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vbd/5/2049 + claim_lock block + local lockdir=/var/run/xen-hotplug/block + mkdir -p /var/run/xen-hotplug + _claim_lock /var/run/xen-hotplug/block + local lockdir=/var/run/xen-hotplug/block ++ _lock_owner /var/run/xen-hotplug/block ++ cat /var/run/xen-hotplug/block/owner ++ echo unknown + local owner=unknown + local retries=0 + '[' 0 -lt 100 ']' + mkdir /var/run/xen-hotplug/block + trap 'release_lock /var/run/xen-hotplug/block; sigerr' ERR + _update_lock_info /var/run/xen-hotplug/block + echo '12842: /etc/xen/scripts/xen-hotplug-cleanup' + return ++ xenstore-read backend/vbd/5/2049/frontend + bridge=xenbr1 + '[' -z xenbr1 ']' + case "$command" in + do_without_error brctl delif xenbr1 vif5.0 + brctl delif xenbr1 vif5.0 + xenstore-rm -t /local/domain/5/device/vbd/2049 + log debug brctl delif xenbr1 'vif5.0 failed' + local level=debug + shift + logger -p daemon.debug -- /etc/xen/scripts/vif-bridge: brctl delif xenbr1 'vif5.0 failed' + do_without_error ifconfig vif5.0 down + ifconfig vif5.0 down + true + xenstore-rm -t backend/vbd/5/2049 + log debug ifconfig vif5.0 'down failed' + local level=debug + shift + logger -p daemon.debug -- /etc/xen/scripts/vif-bridge: ifconfig vif5.0 'down failed' + handle_iptable + iptables -L -n + '[' 10.1.19.20 '!=' '' ']' + local addr + for addr in '"$ip"' + frob_iptable -s 10.1.19.20 + '[' offline == online ']' + local c=-D + iptables -D FORWARD -m physdev --physdev-in vif5.0 -s 10.1.19.20 -j ACCEPT + '[' -D == -D ']' + frob_iptable -p udp --sport 68 --dport 67 + '[' offline == online ']' + local c=-D + iptables -D FORWARD -m physdev --physdev-in vif5.0 -p udp --sport 68 --dport 67 -j ACCEPT + xenstore-rm -t error/backend/vbd/5/2049 + '[' -D == -D ']' + log debug 'Successful vif-bridge offline for vif5.0, bridge xenbr1.' + local level=debug + shift + logger -p daemon.debug -- /etc/xen/scripts/vif-bridge: 'Successful vif-bridge offline for vif5.0, bridge xenbr1.' + '[' offline == online ']' ++ export PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ++ PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ++ export LANG=POSIX ++ LANG=POSIX +++ set +++ grep '^LC_' + true + release_lock block + _release_lock /var/run/xen-hotplug/block + trap sigerr ERR + rm -rf /var/run/xen-hotplug/block +++ cut -d= -f1 ++ unset ++ trap sigerr ERR ++ log debug XENBUS_PATH=backend/vif/5/0 ++ local level=debug ++ shift ++ logger -p daemon.debug -- /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/vif/5/0 + claim_lock block + local lockdir=/var/run/xen-hotplug/block + mkdir -p /var/run/xen-hotplug + _claim_lock /var/run/xen-hotplug/block + local lockdir=/var/run/xen-hotplug/block ++ _lock_owner /var/run/xen-hotplug/block ++ cat /var/run/xen-hotplug/block/owner ++ echo unknown + local owner=unknown + local retries=0 + '[' 0 -lt 100 ']' + mkdir /var/run/xen-hotplug/block + trap 'release_lock /var/run/xen-hotplug/block; sigerr' ERR + _update_lock_info /var/run/xen-hotplug/block + echo '12904: /etc/xen/scripts/xen-hotplug-cleanup' + return ++ xenstore-read backend/vif/5/0/frontend + xenstore-rm -t /local/domain/5/device/vif/0 + true + xenstore-rm -t backend/vif/5/0 + xenstore-rm -t error/backend/vif/5/0 + true + release_lock block + _release_lock /var/run/xen-hotplug/block + trap sigerr ERR + rm -rf /var/run/xen-hotplug/block _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |