Re: [Xen-users] Issue with booting domU

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Sat, 2015-09-12 at 23:05 -0400, Geoffrey Tran wrote:
> Hello,
> 
> I have been trying to use Xen 4.5 in an openstack distribution,
> using libvirt.  It works fine with booting the cirrosOS sample
> images, but does not boot my custom image.  
> 
> The custom image is an Ubuntu 12.04 image, built according to
> the instructions at:
> http://docs.openstack.org/image-guide/content/ubuntu-image.html
> 
> This image was build on a separate server that uses KVM as it's
> hypervisor.  Right now, the instance fails and I have pulled
> the following log from libvirt:
> 
[... logs moved to the end and annotated...]
> 
> It looks like there is a timeout with waiting for "state 6"?

Right. The toolstack seems to have locally attached the disk to dom0 (using
a qdisk backend from qemu) in order to run pygrub, and then has been unable
to unplug it again afterwards.

It would be worth having a look in the log file for the dom0 instance of
qemu (the one started by the xencommons script). I've no idea where that
actually goes though :-/ Perhaps under /var/log/xen somewhere?

Roger and George (Ccd) have both been looking at these loopback mounts
recently in the dev branch, so maybe they know of an issue with 4.5.
> libxl: debug: libxl_create.c:1501:do_domain_create: ao 0x7f1f10001ed0: 
> create: how=(nil) callback=(nil) poller=0x7f1f10000c80
> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk 
> vdev=xvda spec.backend=qdisk
> libxl: debug: libxl_create.c:907:initiate_domain_create: running bootloader
> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk 
> vdev=(null) spec.backend=qdisk
> libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk 
> vdev=xvdb spec.backend=qdisk
> libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f100024e0: deregister unregistered
> libxl: debug: libxl.c:3088:local_device_attach_cb: locally attaching qdisk 
> /dev/xvdb

Here the disk is attached to dom0.

> libxl: debug: libxl_bootloader.c:411:bootloader_disk_attached_cb: Config 
> bootloader value: pygrub
> libxl: debug: libxl_bootloader.c:427:bootloader_disk_attached_cb: Checking 
> for bootloader in libexec path: /usr/lib/xen-4.5/bin/pygrub
> libxl: debug: libxl_create.c:1517:do_domain_create: ao 0x7f1f10001ed0: 
> inprogress: poller=0x7f1f10000c80, flags=i
> libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
> w=0x7f1f100026a0 wpath=/local/domain/6 token=3/0: register slotnum=3
> libxl: debug: libxl_event.c:1945:libxl__ao_progress_report: ao 
> 0x7f1f10001ed0: progress report: callback queued aop=0x7f1f10005950
> libxl: debug: libxl_bootloader.c:537:bootloader_gotptys: executing 
> bootloader: /usr/lib/xen-4.5/bin/pygrub
> libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
> /usr/lib/xen-4.5/bin/pygrub
> libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
> --output=/var/run/xen/bootloader.6.out
> libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
> --output-format=simple0
> libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
> --output-directory=/var/run/xen/bootloader.6.d
> libxl: debug: libxl_bootloader.c:541:bootloader_gotptys:   bootloader arg: 
> /dev/xvdb
> libxl: debug: libxl_event.c:1334:egc_run_callbacks: ao 0x7f1f10001ed0: 
> progress report: callback aop=0x7f1f10005950
> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f100026a0 
> wpath=/local/domain/6 token=3/0: event epath=/local/domain/6
> libxl: debug: libxl_bootloader.c:636:bootloader_finished: bootloader completed
> libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
> output contained kernel /var/run/xen/bootloader.6.d/boot_kernel.azlwht
> libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
> output contained ramdisk /var/run/xen/bootloader.6.d/boot_ramdisk.Z_gkwV
> libxl: debug: libxl_bootloader.c:138:bootloader_result_command: bootloader 
> output contained args root=/dev/mapper/ubuntu--vg-root ro   splash quiet 
> rtsched=gsn_edf $vt_handoff
> libxl: debug: libxl_bootloader.c:649:bootloader_finished: bootloader 
> execution successful

pygrub has run successfully. 

> libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f100026a0 wpath=/local/domain/6 token=3/0: deregister slotnum=3
> libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
> w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: 
> register slotnum=3
> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
> wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
> epath=/local/domain/0/backend/qdisk/0/51728/state
> libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting 
> state 5

Here we are waiting for the device to unplug.

> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
> wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
> epath=/local/domain/0/backend/qdisk/0/51728/state
> libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting 
> state 5
> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
> wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: event 
> epath=/local/domain/0/backend/qdisk/0/51728/state
> libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting 
> state 5
> libxl: debug: libxl_event.c:848:devstate_timeout: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6  timed out
> libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/1: 
> deregister slotnum=3
> libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f10002458: deregister unregistered
> libxl: debug: libxl_event.c:577:libxl__ev_xswatch_register: watch 
> w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: 
> register slotnum=3
> libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x7f1f10002458 
> wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: event 
> epath=/local/domain/0/backend/qdisk/0/51728/state
> libxl: debug: libxl_event.c:834:devstate_watch_callback: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6 still waiting 
> state 5
> libxl: debug: libxl_event.c:848:devstate_timeout: backend 
> /local/domain/0/backend/qdisk/0/51728/state wanted state 6  timed out
> libxl: debug: libxl_event.c:615:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f10002458 wpath=/local/domain/0/backend/qdisk/0/51728/state token=3/2: 
> deregister slotnum=3
> libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f10002458: deregister unregistered
> libxl: error: libxl_device.c:950:device_backend_callback: unable to remove 
> device with path /local/domain/0/backend/qdisk/0/51728
> libxl: debug: libxl_event.c:629:libxl__ev_xswatch_deregister: watch 
> w=0x7f1f100024e0: deregister unregistered
> libxl: error: libxl.c:3168:local_device_detach_cb: unable to remove vbd with 
> id 51728: No such file or directory
> libxl: error: libxl_bootloader.c:278:bootloader_local_detached_cb: unable to 
> detach locally attached disk

And now we have failed to unplug

> libxl: error: libxl_create.c:1121:domcreate_rebuild_done: cannot (re-)build 
> domain: -9
> libxl: debug: libxl_event.c:1765:libxl__ao_complete: ao 0x7f1f10001ed0: 
> complete, rc=-3
> libxl: debug: libxl_event.c:1737:libxl__ao__destroy: ao 0x7f1f10001ed0: 
> destroy
> xc: debug: hypercall buffer: total allocations:66 total releases:66
> xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
> xc: debug: hypercall buffer: cache current size:2
> xc: debug: hypercall buffer: cache hits:52 misses:2 toobig:12


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users