[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] PV guests hang at boot time


  • To: xen-users@xxxxxxxxxxxxxxxxxxx
  • From: Maarten Vanraes <maarten@xxxxx>
  • Date: Fri, 23 Oct 2009 11:32:57 +0200
  • Delivery-date: Fri, 23 Oct 2009 02:34:44 -0700
  • List-id: Xen user discussion <xen-users.lists.xensource.com>

Hello,

i used xen 3.2 before on 2.6.21, now i'm trying 3.4.0 on 2.6.31-4.

my domU's hang at booting time...

I've been reading and looking around for about a week now.

I've identified 3 different places to hang (tested with the same domUs):
 * right after kernel booted, before init
 * after "mounting swap..."
 * after "starting crond"
 * sometimes it boots until login and i can login fine.

i'm alway starting with "xm create -c <domname>"

in the host: i see that the "xm" command is always defunct.

if it's after starting crond. then SSH is also started, and i can ssh fine. if 
it's before, i cannot SSH. (this also happens if i manually start with "xm 
create ", and do xm console later; the only difference is that there is no 
defunct xm process).

I'm using the ebuild from the gentoo-patches and google code with kernel 
2.6.31-r4 .

both dom0 and domU kernels are that kernel, allthough they have different 
configs. (see below)

my domUs are debootstrapped debian4, with udev and syslog-ng installed.

I tried booting also with extra='console=xvc0' and xencons=tty1 in different 
combinations, but that was not it.

I thought it was something console-related, because of the error (see below):
"XENBUS: Device with no driver: device/console/0"

the xen log files show no error, i can figure out.

also something i see, i also tried different things here, but cant find it:
PCI: Fatal: No config space access function found
PCI: setting up Xen PCI frontend stub


xend-debug log does have errors, but i don't think they are serious:
----------------------------------------------------------------------------------------
cat: /sys/bus/scsi/devices/host0/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host0/model: No such file or directory
cat: /sys/bus/scsi/devices/host0/type: No such file or directory
cat: /sys/bus/scsi/devices/host0/rev: No such file or directory
cat: /sys/bus/scsi/devices/host0/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host1/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host1/model: No such file or directory
cat: /sys/bus/scsi/devices/host1/type: No such file or directory
cat: /sys/bus/scsi/devices/host1/rev: No such file or directory
cat: /sys/bus/scsi/devices/host1/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host2/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host2/model: No such file or directory
cat: /sys/bus/scsi/devices/host2/type: No such file or directory
cat: /sys/bus/scsi/devices/host2/rev: No such file or directory
cat: /sys/bus/scsi/devices/host2/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host3/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host3/model: No such file or directory
cat: /sys/bus/scsi/devices/host3/type: No such file or directory
cat: /sys/bus/scsi/devices/host3/rev: No such file or directory
cat: /sys/bus/scsi/devices/host3/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host4/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host4/model: No such file or directory
cat: /sys/bus/scsi/devices/host4/type: No such file or directory
cat: /sys/bus/scsi/devices/host4/rev: No such file or directory
cat: /sys/bus/scsi/devices/host4/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host5/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host5/model: No such file or directory
cat: /sys/bus/scsi/devices/host5/type: No such file or directory
cat: /sys/bus/scsi/devices/host5/rev: No such file or directory
cat: /sys/bus/scsi/devices/host5/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host6/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host6/model: No such file or directory
cat: /sys/bus/scsi/devices/host6/type: No such file or directory
cat: /sys/bus/scsi/devices/host6/rev: No such file or directory
cat: /sys/bus/scsi/devices/host6/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host7/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host7/model: No such file or directory
cat: /sys/bus/scsi/devices/host7/type: No such file or directory
cat: /sys/bus/scsi/devices/host7/rev: No such file or directory
cat: /sys/bus/scsi/devices/host7/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/host8/vendor: No such file or directory
cat: /sys/bus/scsi/devices/host8/model: No such file or directory
cat: /sys/bus/scsi/devices/host8/type: No such file or directory
cat: /sys/bus/scsi/devices/host8/rev: No such file or directory
cat: /sys/bus/scsi/devices/host8/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:0/vendor: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:0/model: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:0/type: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:0/rev: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:0/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:1/vendor: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:1/model: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:1/type: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:1/rev: No such file or directory
cat: /sys/bus/scsi/devices/target8:0:1/scsi_level: No such file or directory
cat: /sys/bus/scsi/devices/target8:1:0/vendor: No such file or directory
cat: /sys/bus/scsi/devices/target8:1:0/model: No such file or directory
cat: /sys/bus/scsi/devices/target8:1:0/type: No such file or directory
cat: /sys/bus/scsi/devices/target8:1:0/rev: No such file or directory
cat: /sys/bus/scsi/devices/target8:1:0/scsi_level: No such file or directory




xm console output:
-------------------------
Using config file "/etc/xen/vm-test6".
Started domain vm-test6 (id=15)
                               [    0.000000] Linux version 2.6.31 
(root@sparco01) (gcc version 4.3.2 (Gentoo 4.3.2-r3 p1.6, pie-10.1.5) ) #15 
SMP Thu Oct 22 16:44:31 CEST 2009
[    0.000000] Command line: root=/dev/sda1 ro console=xvc0 xencons=tty1
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] maxmem of 4104M is invalid for an initial allocation of 128M, 
using 4096M
[    0.000000] Xen-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 0000000100000000 (usable)
[    0.000000] last_pfn = 0x100000 max_arch_pfn = 0x80000000
[    0.000000] init_memory_mapping: 0000000000000000-0000000100000000
[    0.000000] (3 early reservations) ==> bootmem [0000000000 - 0008000000]
[    0.000000]   #0 [0001485000 - 00014d7000]     Xen provided ==> 
[0001485000 - 00014d7000]
[    0.000000]   #1 [0001000000 - 0001464cb8]    TEXT DATA BSS ==> 
[0001000000 - 0001464cb8]
[    0.000000]   #2 [00014d7000 - 0001cdc000]          PGTABLE ==> 
[00014d7000 - 0001cdc000]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x00100000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x00008000
[    0.000000]     0: 0x00100000 -> 0x00100000
[    0.000000] NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Allocated 11 4k pages, static data 41120 bytes
[    0.000000] Swapping MFNs for PFN 13a6 and 1493 (MFN 2606cc and 2605df)
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total 
pages: 18432
[    0.000000] Kernel command line: root=/dev/sda1 ro console=xvc0 
xencons=tty1
[    0.000000] PID hash table entries: 512 (order: 9, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.000000] Initializing CPU#0
[    0.000000] Software IO TLB disabled
[    0.000000] Memory: 52288k/4194304k available (2511k kernel code, 4063232k 
absent, 78716k reserved, 1190k data, 220k init)
[    0.000000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, 
Nodes=1
[    0.000000] NR_IRQS:336
[    0.000000] Xen reported: 2493.748 MHz processor.
[    0.000999] Console: colour dummy device 80x25
[    0.000999] console [tty-1] enabled
[    0.862897] Calibrating delay using timer specific routine.. 4990.76 
BogoMIPS (lpj=2495382)
[    0.862945] Mount-cache hash table entries: 256
[    0.863126] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.863131] CPU: L2 cache: 6144K
[    0.863142] SMP alternatives: switching to UP code
[    0.878077] Freeing SMP alternatives: 22k freed
[    0.878172] Brought up 1 CPUs
[    0.879684] NET: Registered protocol family 16
[    0.880701] Brought up 1 CPUs
[    0.880910] PCI: Fatal: No config space access function found
[    0.880919] PCI: setting up Xen PCI frontend stub
[    0.883166] bio: create slab <bio-0> at 0
[    0.883483] suspend: event channel 9
[    0.883782] xen_mem: Initialising balloon driver.
[    1.018676] SCSI subsystem initialized
[    1.018815] usbcore: registered new interface driver usbfs
[    1.018853] usbcore: registered new interface driver hub
[    1.018893] usbcore: registered new device driver usb
[    1.019198] PCI: System does not support PCI
[    1.019202] PCI: System does not support PCI
[    1.019616] NET: Registered protocol family 2
[    1.019675] IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
[    1.019865] TCP established hash table entries: 16384 (order: 6, 262144 
bytes)
[    1.020169] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[    1.020426] TCP: Hash tables configured (established 16384 bind 16384)
[    1.020431] TCP reno registered
[    1.020489] NET: Registered protocol family 1
[    1.020628] platform rtc_cmos: registered platform RTC device (no PNP 
device found)
[    1.022852] VFS: Disk quotas dquot_6.5.2
[    1.022929] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.023370] msgmni has been set to 256
[    1.023648] alg: No test for stdrng (krng)
[    1.023659] io scheduler noop registered
[    1.023661] io scheduler anticipatory registered
[    1.023664] io scheduler deadline registered
[    1.023672] io scheduler cfq registered (default)
[    1.023722] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    1.023745] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    1.023895] Linux agpgart interface v0.103
[    1.024949] brd: module loaded
[    1.025551] loop: module loaded
[    1.025696] nbd: registered device at major 43
[    1.028481] Xen virtual console successfully installed as tty1
[    1.028525] Event-channel device installed.
[    1.050852] netfront: Initialising virtual ethernet driver.
[    1.067106] xen-vbd: registered block device major 8
[    1.067122] blkfront: sda1: barriers enabled
[    1.067997] blkfront: sdb1: barriers enabled
[    1.076451] usbmon: debugfs is not available
[    1.076454] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.076493] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.076527] uhci_hcd: USB Universal Host Controller Interface driver
[    1.076601] Initializing USB Mass Storage driver...
[    1.076625] usbcore: registered new interface driver usb-storage
[    1.076629] USB Mass Storage support registered.
[    1.076653] usbcore: registered new interface driver libusual
[    1.077546] i8042.c: No controller found.
[    1.077574] mice: PS/2 mouse device common for all mice
[    1.077802] TCP cubic registered
[    1.077804] Initializing XFRM netlink socket
[    1.077809] NET: Registered protocol family 17
[    1.077906] XENBUS: Device with no driver: device/console/0
[    1.152249] kjournald starting.  Commit interval 5 seconds
[    1.152264] EXT3-fs: mounted filesystem with writeback data mode.
[    1.152273] VFS: Mounted root (ext3 filesystem) readonly on device 8:1.
[    1.152351] Freeing unused kernel memory: 220k freed
INIT: version 2.86 booting
Activating swap...[   13.319535] Adding 262136k swap on /dev/sdb1.  
Priority:-1 extents:1 across:262136k SS
done.
Checking root file system...fsck 1.40-WIP (14-Nov-2006)
/dev/sda1: clean, 15166/65808 files, 73647/263168 blocks
done.
[   14.432238] EXT3 FS on sda1, internal journal
Setting the system clock..
Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access method.
Cleaning up ifupdown....
Loading kernel modules...done.
Loading device-mapper support.
Checking file systems...fsck 1.40-WIP (14-Nov-2006)
done.
Setting kernel variables...done.
Mounting local filesystems...done.
Activating swapfile swap...done.
Setting up networking....
Configuring network interfaces...done.
INIT: Entering runlevel: 2
Starting system logging: syslog-ng.
Starting nagios-nrpe: nagios-nrpe.
* Not starting internet superserver: no services enabled.
Starting OpenBSD Secure Shell server: sshd[   22.389110] NET: Registered 
protocol family 10
.
Starting periodic command scheduler: crond.

Debian GNU/Linux 4.0 test6.colo.ba.be tty1

test6.colo.ba.be login: root
Password:
Linux test6.colo.ba.be 2.6.31 #15 SMP Thu Oct 22 16:44:31 CEST 2009 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.


dom0 kernel .config file
-------------------------------

sparco01 xen # zcat /proc/config.gz | grep XEN
CONFIG_X86_64_XEN=y
# CONFIG_XEN_PCIDEV_FRONTEND is not set
CONFIG_XEN=y
CONFIG_XEN_INTERFACE_VERSION=0x00030207
# XEN
CONFIG_XEN_PRIVILEGED_GUEST=y
# CONFIG_XEN_UNPRIVILEGED_GUEST is not set
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_XENBUS_DEV=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_BLKDEV_TAP=y
CONFIG_XEN_BLKDEV_TAP2=y
CONFIG_XEN_BLKBACK_PAGEMAP=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_NETDEV_TX_SHIFT=8
# CONFIG_XEN_NETDEV_PIPELINED_TRANSMITTER is not set
CONFIG_XEN_NETDEV_LOOPBACK=m
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
# CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set
# CONFIG_XEN_PCIDEV_BE_DEBUG is not set
# CONFIG_XEN_TPMDEV_BACKEND is not set
CONFIG_XEN_SCSI_BACKEND=y
CONFIG_XEN_USB_BACKEND=y
# CONFIG_XEN_BLKDEV_FRONTEND is not set
# CONFIG_XEN_NETDEV_FRONTEND is not set
# CONFIG_XEN_SCSI_FRONTEND is not set
# CONFIG_XEN_USB_FRONTEND is not set
# CONFIG_XEN_GRANT_DEV is not set
# CONFIG_XEN_FRAMEBUFFER is not set
CONFIG_XEN_DISABLE_SERIAL=y
CONFIG_XEN_SYSFS=y
CONFIG_XEN_NR_GUEST_DEVICES=256
# CONFIG_XEN_COMPAT_030002_AND_LATER is not set
# CONFIG_XEN_COMPAT_030004_AND_LATER is not set
# CONFIG_XEN_COMPAT_030100_AND_LATER is not set
CONFIG_XEN_COMPAT_030200_AND_LATER=y
# CONFIG_XEN_COMPAT_030300_AND_LATER is not set
# CONFIG_XEN_COMPAT_LATEST_ONLY is not set
CONFIG_XEN_COMPAT=0x030200
CONFIG_XEN_VCPU_INFO_PLACEMENT=y
CONFIG_XEN_SMPBOOT=y
CONFIG_XEN_DEVMEM=y
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y



domU kernel .config file:
--------------------------------
test6:~# zcat /proc/config.gz | grep XEN
CONFIG_X86_64_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
# CONFIG_XEN_PCIDEV_FE_DEBUG is not set
CONFIG_XEN=y
CONFIG_XEN_INTERFACE_VERSION=0x00030207
# XEN
# CONFIG_XEN_PRIVILEGED_GUEST is not set
CONFIG_XEN_UNPRIVILEGED_GUEST=y
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_XENBUS_DEV=y
# CONFIG_XEN_BACKEND is not set
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_ACCEL_SFC_FRONTEND is not set
CONFIG_XEN_SCSI_FRONTEND=y
CONFIG_XEN_USB_FRONTEND=y
# CONFIG_XEN_USB_FRONTEND_HCD_STATS is not set
# CONFIG_XEN_GRANT_DEV is not set
CONFIG_XEN_FRAMEBUFFER=y
CONFIG_XEN_KEYBOARD=y
CONFIG_XEN_DISABLE_SERIAL=y
CONFIG_XEN_SYSFS=y
CONFIG_XEN_NR_GUEST_DEVICES=256
# CONFIG_XEN_COMPAT_030002_AND_LATER is not set
# CONFIG_XEN_COMPAT_030004_AND_LATER is not set
# CONFIG_XEN_COMPAT_030100_AND_LATER is not set
# CONFIG_XEN_COMPAT_030200_AND_LATER is not set
# CONFIG_XEN_COMPAT_030300_AND_LATER is not set
CONFIG_XEN_COMPAT_LATEST_ONLY=y
CONFIG_XEN_COMPAT=0xffffff
CONFIG_XEN_VCPU_INFO_PLACEMENT=y
CONFIG_XEN_SMPBOOT=y
CONFIG_XEN_DEVMEM=y
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y

-- 
BA N.V. - http://www.ba.be

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.