[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Xen 4.4.1 on Debian Jessie: freeze using "xl create" on second VM





> Subject: Re: [Xen-users] Xen 4.4.1 on Debian Jessie: freeze using "xl create" on second VM
> To: badkins79@xxxxxxx; xen-users@xxxxxxxxxxxxx
> From: roger.pau@xxxxxxxxxx
> Date: Tue, 23 Feb 2016 09:23:29 +0100
>
> El 22/2/16 a les 20:11, Brandon Adkins ha escrit:
> >
> >> El 22/2/16 a les 18:40, Brandon Adkins ha escrit:
> >>>
> >>>>
> >>>> El 19/2/16 a les 17:38, Brandon Adkins ha escrit:
> >>>>> I am having issues migrating my Xen setup from Debian Wheezy to Debian Jessie. The errors I see are not very concrete, so it doesn't give me much details.Here is what I am seeing:First, when I make a new Xen VM and connect to its console, I see "xenconsole: Could not open tty `': No such file or directory". It seems to boot fine after that, but after returning to Dom0, the console is kinda jacked up. This mimics exactly the issue described athttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794071 . Apparently there is a fix as of Oct 2015, but just not released on Jessie yet? Will that be released on Jessie sometime soon?Second, and this is the bigger issue: I can make 1 VM like this fine, but when I try to make a second one, it just seems to hang. If I use "xl create -c" so I see the console the whole time, I see the bootloader menu appear, and when the kernel selection is finished, it prints the error about the tty, and then nothing else. I don't know if the tty issue is rela
> te
> >> d
> >>>> at all."xl list" shows the second VM in the list, but it is marked as only having 1 vcpu. If I try to change this with "xl vcpu-set 2" (or any number higher), it is just ignored. I can't find any log files to indicate there is a problem at all. The VM just doesn't start. It was created from the same template as the first VM, so VM1 and VM2 are basically copies of each other, yet only the first one will run.I have no idea if I am doing something wrong or if this is a bug in Jessie. I will say that everything that I am doing now has worked perfectly for years using Debian Wheezy. The only change I needed was to start using "xl" instead of "xm".Anyone have clues on figuring this out?Thanks,Beau
> >>>>
> >>>> Hello,
> >>>>
> >>>> Can you please paste the config file of those two VMs?
> >>>>
> >>>> Also, can you paste the output when creating them with:
> >>>>
> >>>> # xl -vvv create <config.cfg>
> >>>>
> >>>> Are there any errors displayed by Xen or Linux in the serial console or
> >>>> dmesg?
> >>>>
> >>>> Roger.
> >>>
> >>
> >> Hello,
> >>
> >> I'm sorry but your MUA seems to completely mangle the messages, and I
> >> cannot parse the output you posted. Could you try to use a different MUA
> >> that adds proper line breaks?
> >>
> >> Thanks, Roger.
> >>
> > Trying again
> >
> > Hi Roger,
> > Thanks for the help. Here are the 2 config files:guest0:bootloader = '/usr/lib/xen-4.4/bin/pygrub'vcpus = '6'cpus = '2-7'memory = '6000'root = '/dev/xvda2 ro'disk = [ 'phy:/dev/xenvolgroup/guest0disk,xvda2,w', 'phy:/dev/xenvolgroup/guest0swap,xvda1,w' ]name = 'guest0'vif = [ 'mac=AE:57:EC:01:54:8C,bridge=xenbr0' ]-----------------------------------------guest1:bootloader = '/usr/lib/xen-4.4/bin/pygrub'vcpus = '6'cpus = '2-7'memory = '6000'root = '/dev/xvda2 ro'disk = [ 'phy:/dev/xenvolgroup/guest1disk,xvda2,w', 'phy:/dev/xenvolgroup/guest1swap,xvda1,w' ]name = 'guest1'vif = [ 'mac=76:7B:D6:6F:34:C6,bridge=xenbr0' ]---------------------------------------------Output of xl -vvv create:libxl: debug: libxl_create.c:1342:do_domain_create: ao 0x22ff8b0: create: how=(nil) callback=(nil) poller=0x22ff9b0libxl: debug: libxl_device.c:251:libxl__device_disk_set_backen
> d: Disk vdev=xvda2 spec.backend=unknownlibxl: debug: libxl_device.c:286:libxl__device_disk_set_backend: Disk vdev=xvda2, using backend phylibxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=unknownlibxl: debug: libxl_device.c:286:libxl__device_disk_set_backend: Disk vdev=xvda1, using backend phylibxl: debug: libxl_create.c:797:initiate_domain_create: running bootloaderlibxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=(null) spec.backend=phylibxl: debug: libxl.c:2634:libxl__device_disk_local_initiate_attach: locally attaching PHY disk /dev/xenvolgroup/guest1disklibxl: debug: libxl_bootloader.c:409:bootloader_disk_attached_cb: Config bootloader value: /usr/lib/xen-4.4/bin/pygrublibxl: debug: libxl_create.c:1356:do_domain_create: ao 0x22ff8b0: inprogress: poller=0x22ff9b0, flags=ilibxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x22f5178 wpath=/local/domain/127 token=3/0: register slotnum=3libxl: debug:
> libxl_event.c:1761:libxl__ao_progress_report: ao 0x22ff8b0: progress report: ignoredlibxl: debug: libxl_bootloader.c:535:bootloader_gotptys: executing bootloader: /usr/lib/xen-4.4/bin/pygrublibxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: /usr/lib/xen-4.4/bin/pygrublibxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: --args=root=/dev/xvda2 rolibxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: --output=/var/run/xen/bootloader.127.outlibxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: --output-format=simple0libxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: --output-directory=/var/run/xen/bootloader.127.dlibxl: debug: libxl_bootloader.c:539:bootloader_gotptys: bootloader arg: /dev/xenvolgroup/guest1disklibxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x22f5178 wpath=/local/domain/127 token=3/0: event epath=/local/domain/127libxl: debug: libxl_bootloader.c:634
> :bootloader_finished: bootloader completedlibxl: debug: libxl_bootloader.c:136:bootloader_result_command: bootloader output contained kernel /var/run/xen/bootloader.127.d/boot_kernel.m3pOqrlibxl: debug: libxl_bootloader.c:136:bootloader_result_command: bootloader output contained ramdisk /var/run/xen/bootloader.127.d/boot_ramdisk.DQT0dDlibxl: debug: libxl_bootloader.c:136:bootloader_result_command: bootloader output contained args root=/dev/xvda2 ro crashkernel=auto root=/dev/xvda2 rolibxl: debug: libxl_bootloader.c:647:bootloader_finished: bootloader execution successfullibxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x22f5178wpath=/local/domain/127 token=3/0: deregister slotnum=3domainbuilder: detail: xc_dom_allocate: cmdline="root=/dev/xvda2 ro crashkernel=auto root=/dev/xvda2 ro ", features="(null)"libxl: debug: libxl_dom.c:364:libxl__build_pv: pv kernel mapped 1 path /var/run/xen/bootloader.127.d/boot_kernel.m3pOqrdomainbuilder: detail: xc_dom_kernel_mem: ca
> lleddomainbuilder: detail: xc_dom_ramdisk_mem: calleddomainbuilder: detail: xc_dom_boot_xen_init: ver 4.4, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64domainbuilder: detail: xc_dom_parse_image: calleddomainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...domainbuilder: detail: loader probe faileddomainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...domainbuilder: detail: xc_dom_malloc : 8861 kBdomainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x3e7bbc -> 0x8a7564domainbuilder: detail: loader probe OKxc: detail: elf_parse_binary: phdr: paddr=0x400000 memsz=0x66f000xc: detail: elf_parse_binary: phdr: paddr=0xa6f000 memsz=0x381000xc: detail: elf_parse_binary: memory: 0x400000 -> 0xdf0000xc: detail: elf_xen_parse_note: GUEST_OS = "linux"xc: detail: elf_xen_parse_note: GUEST_VERSION = "2.6"xc: detail: elf_xen_parse_note: XEN_VERSION = "xen-3.0"xc: detail: elf_xen_parse_note: VIRT_BASE = 0xc0000000xc:
> detail: elf_xen_parse_note: ENTRY = 0xc0b11000xc: detail: elf_xen_parse_note: HYPERCALL_PAGE = 0xc0402000xc: detail: elf_xen_parse_note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"xc: detail: elf_xen_parse_note: PAE_MODE = "yes"xc: detail: elf_xen_parse_note: LOADER = "generic"xc: detail: elf_xen_parse_note: unknown xen elf note (0xd)xc: detail: elf_xen_parse_note: SUSPEND_CANCEL = 0x1xc: detail: elf_xen_parse_note: HV_START_LOW = 0xf5800000xc: detail: elf_xen_parse_note: PADDR_OFFSET = 0x0xc: detail: elf_xen_addr_calc_check: addresses:xc: detail: virt_base = 0xc0000000xc: detail: elf_paddr_offset = 0x0xc: detail: virt_offset = 0xc0000000xc: detail: virt_kstart = 0xc0400000xc: detail: virt_kend = 0xc0df0000xc: detail: virt_entry = 0xc0b11000xc: detail: p2m_base = 0xffffffffffffffffdomainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_32p: 0xc0400000 -> 0xc0df0000domainbuilder: detail: xc_dom_mem_init:
> mem 6000 MB, pages 0x177000 pages, 4k eachdomainbuilder: detail: xc_dom_mem_init: 0x177000 pagesdomainbuilder: detail: xc_dom_boot_mem_init: calleddomainbuilder: detail: x86_compat: guest xen-3.0-x86_32p, address size 32domainbuilder: detail: xc_dom_malloc : 12000 kBdomainbuilder: detail: xc_dom_build_image: calleddomainbuilder: detail: xc_dom_alloc_segment: kernel : 0xc0400000 -> 0xc0df0000 (pfn 0x400 + 0x9f0 pages)domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x400+0x9f0 at 0x7fd07b610000xc: detail: elf_load_binary: phdr 0 at 0x7fd07b610000 -> 0x7fd07bc7f000xc: detail: elf_load_binary: phdr 1 at 0x7fd07bc7f000 -> 0x7fd07bdb2000domainbuilder: detail: xc_dom_alloc_segment: ramdisk : 0xc0df0000 -> 0xc2ee0000 (pfn 0xdf0 + 0x20f0 pages)domainbuilder: detail: xc_dom_malloc : 197 kBdomainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xdf0+0x20f0 at 0x7fd079520000domainbuilder: detail: xc_dom_do_gunzip: unzip ok,
> 0xd262fe -> 0x20ef410domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xc2ee0000 -> 0xc34bc000 (pfn 0x2ee0 + 0x5dc pages)domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x2ee0+0x5dc at 0x7fd0800d3000domainbuilder: detail: xc_dom_alloc_page : start info : 0xc34bc000 (pfn 0x34bc)domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xc34bd000 (pfn 0x34bd)domainbuilder: detail: xc_dom_alloc_page : console : 0xc34be000 (pfn 0x34be)domainbuilder: detail: nr_page_tables: 0x00000000ffffffff/32: 0x0000000000000000 -> 0x00000000ffffffff, 1 table(s)domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x00000000c0000000 -> 0x00000000ffffffff, 1 table(s)domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x00000000c0000000 -> 0x00000000c37fffff, 28 table(s)domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xc34bf000 -> 0xc34dd000 (pfn 0x34bf + 0x1e pages)domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: do
> mU mapping: pfn 0x34bf+0x1e at 0x7fd084ad6000domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xc34dd000 (pfn 0x34dd)domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xc34de000domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc3800000domainbuilder: detail: xc_dom_boot_image: calleddomainbuilder: detail: arch_setup_bootearly: doing nothingdomainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p <= matchesdomainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32pdomainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64domainbuilder: detail: xc_dom_update_guest_p2m: dst 32bit, pages 0x177000domainbuilder: detail: clear_page: pfn 0x34be, mfn 0x17cd7adomainbuilder: detail: clear_page: pfn 0x34bd, mfn 0x17cd7bdomainbuilder:
> detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x34bc+0x1at 0x7fd084b31000domainbuilder: detail: start_info_x86_32: calleddomainbuilder: detail: setup_hypercall_page: vaddr=0xc0402000 pfn=0x402domainbuilder: detail: domain builder memory footprintdomainbuilder: detail: allocateddomainbuilder: detail: malloc : 21157 kBdomainbuilder: detail: anon mmap : 0 bytesdomainbuilder: detail: mappeddomainbuilder: detail: file mmap : 0 bytesdomainbuilder: detail: domU mmap : 48 MBdomainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xbf615domainbuilder: detail: shared_info_x86_32: calleddomainbuilder: detail: vcpu_x86_32: calleddomainbuilder: detail: vcpu_x86_32: cr3: pfn 0x34bf mfn 0x17cd79domainbuilder: detail: launch_vm: called, ctxt=0x7fd084ad4004domainbuilder: detail: xc_dom_release: calledlibxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phylibxl: debug: li
> bxl_event.c:570:libxl__ev_xswatch_register: watch w=0x22f6068 wpath=/local/domain/0/backend/vbd/127/51714/state token=3/1: register slotnum=3libxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=phylibxl: debug: libxl_device.c:251:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=phylibxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x22f7df8 wpath=/local/domain/0/backend/vbd/127/51713/state token=2/2: register slotnum=2libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x22f6068 wpath=/local/domain/0/backend/vbd/127/51714/state token=3/1: event epath=/local/domain/0/backend/vbd/127/51714/statelibxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vbd/127/51714/state wanted state 2 oklibxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x22f6068wpath=/local/domain/0/backend/vbd/127/51714/state token=3/1: deregister slotnum=3libxl: debug: libxl_event.c:618:libxl
> __ev_xswatch_deregister: watch w=0x22f6068: deregister unregisteredlibxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/block addlibxl: debug: libxl_event.c:483:watchfd_callback: watch epath=/local/domain/0/backend/vbd/127/51714/state token=3/1: empty slotlibxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x22f7df8 wpath=/local/domain/0/backend/vbd/127/51713/state token=2/2: event epath=/local/domain/0/backend/vbd/127/51713/statelibxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vbd/127/51713/state wanted state 2 oklibxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x22f7df8wpath=/local/domain/0/backend/vbd/127/51713/state token=2/2: deregister slotnum=2libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22f7df8: deregister unregisteredlibxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/block addlibxl: debug: libxl_event.c:483
> :watchfd_callback: watch epath=/local/domain/0/backend/vbd/127/51713/state token=2/2: empty slotlibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22f60f0: deregister unregisteredlibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22f60f0: deregister unregisteredlibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22f7e80: deregister unregisteredlibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22f7e80: deregister unregisteredlibxl: debug: libxl_event.c:570:libxl__ev_xswatch_register: watch w=0x22fa7c8 wpath=/local/domain/0/backend/vif/127/0/state token=2/3: register slotnum=2libxl: debug: libxl_event.c:514:watchfd_callback: watch w=0x22fa7c8 wpath=/local/domain/0/backend/vif/127/0/state token=2/3: event epath=/local/domain/0/backend/vif/127/0/statelibxl: debug: libxl_event.c:657:devstate_watch_callback: backend /local/domain/0/backend/vif/127/0/state wanted state 2 still waiting state 1libxl: debug: lib
> xl_event.c:514:watchfd_callback: watch w=0x22fa7c8 wpath=/local/domain/0/backend/vif/127/0/state token=2/3: event epath=/local/domain/0/backend/vif/127/0/statelibxl: debug: libxl_event.c:653:devstate_watch_callback: backend /local/domain/0/backend/vif/127/0/state wanted state 2 oklibxl: debug: libxl_event.c:606:libxl__ev_xswatch_deregister: watch w=0x22fa7c8wpath=/local/domain/0/backend/vif/127/0/state token=2/3: deregister slotnum=2libxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22fa7c8: deregister unregisteredlibxl: debug: libxl_device.c:1023:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge onlinelibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22fa850: deregister unregisteredlibxl: debug: libxl_event.c:618:libxl__ev_xswatch_deregister: watch w=0x22fa850: deregister unregisteredlibxl: debug: libxl_event.c:1761:libxl__ao_progress_report: ao 0x22ff8b0: progress report: ignoredlibxl: debug: libxl_event.c:1591:libxl__ao_c
> omplete: ao 0x22ff8b0: complete, rc=0libxl: debug: libxl_event.c:1563:libxl__ao__destroy: ao 0x22ff8b0: destroyxc: debug: hypercall buffer: total allocations:470 total releases:470xc: debug: hypercall buffer: current allocations:0 maximum allocations:3xc: debug: hypercall buffer: cache current size:2xc: debug: hypercall buffer: cache hits:462 misses:2 toobig:6Parsing config from guest.cfg-----------------------------------------------Looking into dmesg, the only interesting thing I see related to this are these 2 lines everytime I try to start the VM: xen:balloon: Cannot add additional memory (-17) xen:balloon: Cannot add additional memory (-17)
> > Thanks!Beau
>
> Hello,
>
> Can you also post the output of xl info before creating the guests?
> AFAICT you are
> giving each guest 6000MB of RAM; are you sure you have enough RAM for
> Dom0 plus this two guests?
>
> From the output you provided, I can only see the trace of one guest
> booting [0] (one execution of xl), what happens with the other guest?
>
> Roger.
>
> [0]
> http://lists.xenproject.org/archives/html/xen-users/2016-02/msg00116.html
>

Roger, yes thats right. Both guests get 6000MB of RAM. The system has 16GB of RAM, and I have allocated 4GB to the host.

Guest0 boots and operates perfectly. It is only guest1 that has issues. It does act like maybe there isnt enough RAM for the second VM, but from my calculations, it looks like it should be fine. Here is the output of "xl info" with no guests running:
host                   : lpwtest
release                : 3.16.0-4-amd64
version                : #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17)
machine                : x86_64
nr_cpus                : 8
max_cpu_id             : 7
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 3292
hw_caps                : bfebfbff:28100800:00000000:00007f00:77bae3ff:00000000:00000001:00000281
virt_caps              : hvm hvm_directio
total_memory           : 16353
free_memory            : 12040
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 4
xen_extra              : .1
xen_version            : 4.4.1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          :
xen_commandline        : placeholder dom0_mem=4096M,max:4096M
cc_compiler            : gcc (Debian 4.9.2-10) 4.9.2
cc_compile_by          : ultrotter
cc_compile_domain      : debian.org
cc_compile_date        : Wed Nov 25 15:46:22 EET 2015
xend_config_format     : 4

Best,
Beau
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.