[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-users] HVM DumU + Stubdom + stoarge backend domain
Hi people, I'm running several virtual machines on a small cluster. Among them are Windows VMs. As a general concept all storage is managed by dedicated storage domU's. In the past we simply mapped a volume from the storage domU to dom0 so that qemu can access it and provide it to windows. Unfortunately this has the major drawback, that when the storage domU crashes or has to be rebooted, the dom0 basically also dies (it locks up so much that you cannot start/stop VMs any more). Now in order to solve this problem, it appears that one has to use a stubdom which runs qemu and thus move the dependency to the starge dumU from dom0 to the stubdom. I think I'm quite far in making it, but when the Windows VM starts, the bios tells me that the disk is not bootable/cannot be read - which I take as a hint that there is simply no disk attached. Checking "xl block-list ..." also seems to conform this. First off: is such a setup even possible? I read only hints on the internet that this should be possible, but I never found an actual working example. My setup: Dom0: debian 8; xen 4.4.1 DomU/storage: debian 8; tools installed; also serves a s a working storage backend for other PV domains DomU/Windows: Windows 2012 Server x64; XEN PV drivers installed DomU/Windows config: -------------- snip ------------ name = "v2-jw-winsvr2012x64-1" builder = "hvm" viridian = 1 memory = 16384 vcpus = 16 device_model_version = "qemu-xen-traditional" device_model_stubdomain_override = 1 boot="c" # Network devices # docs/misc/xl-network-configuration.markdown vif = [ 'mac=00:16:3E:ec:10:e9,bridge=br0,type=vif', 'mac=00:16:3E:53:60:83,bridge=br1,type=vif' ] # Disk Devices # docs/misc/xl-disk-configuration.txt disk = [ 'format=raw,vdev=xvda,backend=osd,target=/dev/osd/v2-jw-winsvr2012x64-1' ] #disk = [ 'format=raw,vdev=xvda,target=/dev/xvda' ] # Guest VGA console configuration, either SDL or VNC vnc = 1 vga = "stdvga" #keymap = "de-de" vncunused = 1 vnclisten = "0.0.0.0" -------------- snip ------------ What I did so far: 1. Installed XEN via apt-get 2. Configured storage backend domain 3. Tested storage backend domain with other PV (linux) DomUs - working 4. Created windows domU with disk provided by dom0 - works 5. Compiled XEN 4.4.1 from souce to get access to the stubdom (could not find a package for it). 6. Changed config of windows domU to use stubdom, and disk still from dom0 - also works 7. Changed config to use storage domU - does not work anymore. Output from xl -vvvvvvvvvv create v2-jw-winsvr2012x64-1.new: -------------- snip ------------ Parsing config from v2-jw-winsvr2012x64-1.new libxl: debug: libxl_create.c:1342:do_domain_create: ao 0x1df4c40: create: how=(nil) callback=(nil) poller=0x1df5670 libxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=unknown libxl: debug: libxl_device.c:183:disk_try_backend: Disk vdev=xvda, is using a storage driver domain, skipping physical device check libxl: debug: libxl_device.c:286:libxl__device_disk_set_backend: Disk vdev=xvda, using backend phy libxl: debug: libxl_create.c:797:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:321:libxl__bootloader_run: not a PV domain, skipping bootloader libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1df59f8: deregister unregistered libxl: debug: libxl_numa.c:478:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=20, free_memkb=34153 libxl: detail: libxl_dom.c:195:numa_place_domain: NUMA placement candidate with 1 nodes, 20 cpus and 34153 KB free selected xc: detail: elf_parse_binary: phdr: paddr=0x100000 memsz=0x9e6a4 xc: detail: elf_parse_binary: memory: 0x100000 -> 0x19e6a4 xc: detail: VIRTUAL MEMORY ARRANGEMENT: Loader: 0000000000100000->000000000019e6a4 Modules: 0000000000000000->0000000000000000 TOTAL: 0000000000000000->00000003ff800000 ENTRY ADDRESS: 0000000000100000 xc: detail: PHYSICAL MEMORY ALLOCATION: 4KB PAGES: 0x0000000000000200 2MB PAGES: 0x00000000000003fb 1GB PAGES: 0x000000000000000e xc: detail: elf_load_binary: phdr 0 at 0x7f90856ad000 -> 0x7f908574252d libxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=phy libxl: debug: libxl_device.c:183:disk_try_backend: Disk vdev=xvda, is using a storage driver domain, skipping physical device check libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1df7488 wpath=/local/domain/1/backend/vbd/43/51712/state token=3/0: register slotnum=3 libxl: debug: libxl_create.c:1356:do_domain_create: ao 0x1df4c40: inprogress: poller=0x1df5670, flags=i libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1df7488 wpath=/local/domain/1/backend/vbd/43/51712/state token=3/0: event epath=/local/domain/1/backend/vbd/43/51712/state libxl: debug: libxl_event.c:657:devstate_watch_callback: backend /local/domain/1/backend/vbd/43/51712/state wanted state 2 still waiting state 1 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1df7488 wpath=/local/domain/1/backend/vbd/43/51712/state token=3/0: event epath=/local/domain/1/backend/vbd/43/51712/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/1/backend/vbd/43/51712/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1df7488 wpath=/local/domain/1/backend/vbd/43/51712/state token=3/0: deregister slotnum=3 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1df7488: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1df7510: deregister unregistered libxl: debug: libxl_numa.c:478:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=7, free_memkb=775 libxl: detail: libxl_dom.c:195:numa_place_domain: NUMA placement candidate with 1 nodes, 20 cpus and 775 KB free selected domainbuilder: detail: xc_dom_allocate: cmdline=" -d 43", features="" libxl: debug: libxl_dom.c:357:libxl__build_pv: pv kernel mapped 0 path /usr/lib/xen/boot/ioemu-stubdom.gz domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/ioemu-stubdom.gz" domainbuilder: detail: xc_dom_malloc_filemap : 2575 kB domainbuilder: detail: xc_dom_malloc : 14244 kB domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x283df3 -> 0xde9080 domainbuilder: detail: xc_dom_boot_xen_init: ver 4.4, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x560828 xc: detail: elf_parse_binary: memory: 0x0 -> 0x560828 xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic" xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS" xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0" xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0" xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0" xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2" xc: detail: elf_xen_parse_guest_info: LOADER="generic" xc: detail: elf_xen_addr_calc_check: addresses: xc: detail: virt_base = 0x0 xc: detail: elf_paddr_offset = 0x0 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0x0 xc: detail: virt_kend = 0x560828 xc: detail: virt_entry = 0x0 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x560828 domainbuilder: detail: xc_dom_mem_init: mem 32 MB, pages 0x2000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x2000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64 domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x561000 (pfn 0x0 + 0x561 pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x0+0x561 at 0x7f9082c60000 xc: detail: elf_load_binary: phdr 0 at 0x7f9082c60000 -> 0x7f90831c0828 domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x561000 -> 0x571000 (pfn 0x561 + 0x10 pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x561+0x10 at 0x7f908573c000 domainbuilder: detail: xc_dom_alloc_page : start info : 0x571000 (pfn 0x571) domainbuilder: detail: xc_dom_alloc_page : xenstore : 0x572000 (pfn 0x572) domainbuilder: detail: xc_dom_alloc_page : console : 0x573000 (pfn 0x573) domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x00000000007fffff, 4 table(s) domainbuilder: detail: xc_dom_alloc_segment: page tables : 0x574000 -> 0x57b000 (pfn 0x574 + 0x7 pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x574+0x7 at 0x7f9085735000 domainbuilder: detail: xc_dom_alloc_page : boot stack : 0x57b000 (pfn 0x57b) domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x57c000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x800000 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: arch_setup_bootearly: doing nothing domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x2000 domainbuilder: detail: clear_page: pfn 0x573, mfn 0x105397f domainbuilder: detail: clear_page: pfn 0x572, mfn 0x1053980 domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x571+0x1 at 0x7f9085754000 domainbuilder: detail: start_info_x86_64: called domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2 domainbuilder: detail: domain builder memory footprint domainbuilder: detail: allocated domainbuilder: detail: malloc : 14343 kB domainbuilder: detail: anon mmap : 0 bytes domainbuilder: detail: mapped domainbuilder: detail: file mmap : 2575 kB domainbuilder: detail: domU mmap : 5604 kB domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0x73a82 domainbuilder: detail: shared_info_x86_64: called domainbuilder: detail: vcpu_x86_64: called domainbuilder: detail: vcpu_x86_64: cr3: pfn 0x574 mfn 0x105397e domainbuilder: detail: launch_vm: called, ctxt=0x7f9085733004 domainbuilder: detail: xc_dom_release: called libxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=phy libxl: debug: libxl_device.c:183:disk_try_backend: Disk vdev=xvda, is using a storage driver domain, skipping physical device check libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1dfa9b8 wpath=/local/domain/1/backend/vbd/44/51712/state token=3/1: register slotnum=3 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1dfa9b8 wpath=/local/domain/1/backend/vbd/44/51712/state token=3/1: event epath=/local/domain/1/backend/vbd/44/51712/state libxl: debug: libxl_event.c:657:devstate_watch_callback: backend /local/domain/1/backend/vbd/44/51712/state wanted state 2 still waiting state 1 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1dfa9b8 wpath=/local/domain/1/backend/vbd/44/51712/state token=3/1: event epath=/local/domain/1/backend/vbd/44/51712/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/1/backend/vbd/44/51712/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1dfa9b8 wpath=/local/domain/1/backend/vbd/44/51712/state token=3/1: deregister slotnum=3 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dfa9b8: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dfaa40: deregister unregistered libxl: debug: libxl_dm.c:1303:libxl__spawn_local_dm: Spawning device-model /usr/lib/xen/bin/qemu-dm with arguments: libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: /usr/lib/xen/bin/qemu-dm libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: -d libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: 44 libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: -domain-name libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: v2-jw-winsvr2012x64-1-dm libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: -vnc libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: 0.0.0.0:0,password libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: -vncunused libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: -M libxl: debug: libxl_dm.c:1305:libxl__spawn_local_dm: xenpv libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1df6070 wpath=/local/domain/0/device-model/44/state token=3/2: register slotnum=3 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1df6070 wpath=/local/domain/0/device-model/44/state token=3/2: event epath=/local/domain/0/device-model/44/state libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1df6070 wpath=/local/domain/0/device-model/44/state token=3/2: event epath=/local/domain/0/device-model/44/state libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1df6070 wpath=/local/domain/0/device-model/44/state token=3/2: deregister slotnum=3 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1df6070: deregister unregistered libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1dfe858 wpath=/local/domain/0/backend/vif/44/0/state token=3/3: register slotnum=3 libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1dff1f8 wpath=/local/domain/0/backend/vif/44/1/state token=2/4: register slotnum=2 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1dfe858 wpath=/local/domain/0/backend/vif/44/0/state token=3/3: event epath=/local/domain/0/backend/vif/44/0/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vif/44/0/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1dfe858 wpath=/local/domain/0/backend/vif/44/0/state token=3/3: deregister slotnum=3 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dfe858: deregister unregistered libxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online libxl: debug: libxl_event.c:483:watchfd_callback: watch epath=/local/domain/0/backend/vif/44/0/state token=3/3: empty slot libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1dff1f8 wpath=/local/domain/0/backend/vif/44/1/state token=2/4: event epath=/local/domain/0/backend/vif/44/1/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vif/44/1/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1dff1f8 wpath=/local/domain/0/backend/vif/44/1/state token=2/4: deregister slotnum=2 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dff1f8: deregister unregistered libxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online libxl: debug: libxl_event.c:483:watchfd_callback: watch epath=/local/domain/0/backend/vif/44/1/state token=2/4: empty slot libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dfe8e0: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dfe8e0: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dff280: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1dff280: deregister unregistered libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1e00738 wpath=/local/domain/0/backend/vif/43/0/state token=2/5: register slotnum=2 libxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x1e01288 wpath=/local/domain/0/backend/vif/43/1/state token=3/6: register slotnum=3 libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1e00738 wpath=/local/domain/0/backend/vif/43/0/state token=2/5: event epath=/local/domain/0/backend/vif/43/0/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vif/43/0/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1e00738 wpath=/local/domain/0/backend/vif/43/0/state token=2/5: deregister slotnum=2 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e00738: deregister unregistered libxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online libxl: debug: libxl_event.c:483:watchfd_callback: watch epath=/local/domain/0/backend/vif/43/0/state token=2/5: empty slot libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x1e01288 wpath=/local/domain/0/backend/vif/43/1/state token=3/6: event epath=/local/domain/0/backend/vif/43/1/state libxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vif/43/1/state wanted state 2 ok libxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x1e01288 wpath=/local/domain/0/backend/vif/43/1/state token=3/6: deregister slotnum=3 libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e01288: deregister unregistered libxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online libxl: debug: libxl_event.c:483:watchfd_callback: watch epath=/local/domain/0/backend/vif/43/1/state token=3/6: empty slot libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e007c0: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e007c0: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e01310: deregister unregistered libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x1e01310: deregister unregistered libxl: debug: libxl_event.c:1761:libxl__ao_progress_report: ao 0x1df4c40: progress report: ignored libxl: debug: libxl_event.c:1591:libxl__ao_complete: ao 0x1df4c40: complete, rc=0 libxl: debug: libxl_event.c:1563:libxl__ao__destroy: ao 0x1df4c40: destroy xc: debug: hypercall buffer: total allocations:1715 total releases:1715 xc: debug: hypercall buffer: current allocations:0 maximum allocations:4 xc: debug: hypercall buffer: cache current size:4 xc: debug: hypercall buffer: cache hits:1697 misses:4 toobig:14 -------------- snip ------------ xl list: v2-jw-winsvr2012x64-1 45 16383 1 r----- 12.8 v2-jw-winsvr2012x64-1-dm 46 32 1 -b---- 0.1 xl block-list v2-jw-winsvr2012x64-1: Vdev BE handle state evt-ch ring-ref BE-path xl block-list v2-jw-winsvr2012x64-1-dm: Vdev BE handle state evt-ch ring-ref BE-path Any ideas what could be the problem? I tried also various vdev-names (hda, xvda, xvdb) - no luck. Any suggestions/hints would be very welcome. Thanks & Regards Simon _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx https://lists.xen.org/xen-users
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |