Re: [Xen-devel] Nested virtualization instabilities

[Etzion Bar-Noy <etzion@xxxxxxxxxxxx>]

Sorry for spamming. Actually - it is a bit different - two results which appear (on XenServer and on VMware). The previous messages were for Vmware. The one below is for XenServer:

xl dmesg:

(XEN) stdvga.c:151:d10 leaving stdvga
(XEN) common.c:3525:d0 tracking VRAM f0000 - f0800
(XEN) stdvga.c:147:d10 entering stdvga and caching modes
(XEN) realmode.c:115:d10 Failed to emulate insn.
(XEN) realmode.c:167:d10 Real-mode emulation failed @ f000:0000fff0: 00 90 23 00 c0 82
(XEN) domain_crash called from realmode.c:168
(XEN) Domain 10 (vcpu#0) crashed on cpu#8:
(XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    8
(XEN) RIP:    f000:[<000000000000fff0>]
(XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest
(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000000000000
(XEN) rdx: 00000000078bfbff   rsi: 0000000000000000   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: 0000000000000000   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000030   cr4: 0000000000002040
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: f000
(XEN) <vm_launch_fail> error code 7
(XEN) domain_crash_sync called from vmcs.c:1293
(XEN) Domain 9 (vcpu#0) crashed on cpu#5:
(XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    5
(XEN) RIP:    0000:[<0000000000000000>]
(XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest
(XEN) rax: 0000000000000000   rbx: ffff830824f90000   rcx: ffff830824f97f80
(XEN) rdx: ffff82d0801d0cb9   rsi: 0000000000000000   rdi: ffff82d0801de7ec
(XEN) rbp: ffff82d080105ab1   rsp: 0000000000000000   r8:  0000000000000004
(XEN) r9:  ffff82d080105b20   r10: ffff830824f97f70   r11: 0000000000000000
(XEN) r12: ffff830824f97f50   r13: ffff83080191c000   r14: 0000000000000004
(XEN) r15: ffff82d080187e15   cr0: 0000000000000039   cr4: 0000000000002050
(XEN) cr3: 00000000feffa000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: 0000
(XEN) grant_table.c:289:d0 Increased maptrack size to 2 frames
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 11, error -22

Thanks
Etzion

On Thu, Dec 19, 2013 at 9:40 PM, Etzion Bar-Noy <etzion@xxxxxxxxxxxx> wrote:

Hi.

With the added flag suggested by you, the VM (nested hypervisor) crashed. Now ESXi will attempt to start the VM and crash as well (unlike before, where it just failed to start it with 'unknown error').

XenServer 6.2 still crashes, just the same.

I am attempting to deploy a Centos6 machine (and clone it) so that I will be able to test both KVM and Xen community (latest stable RPM release for Centos6).

xl dmesg looks much like before:

(XEN) stdvga.c:151:d12 leaving stdvga
(XEN) common.c:3525:d0 tracking VRAM f0000 - f0800
(XEN) stdvga.c:147:d12 entering stdvga and caching modes
(XEN) realmode.c:115:d12 Failed to emulate insn.
(XEN) realmode.c:167:d12 Real-mode emulation failed @ f000:0000fff0: 00 10 23 00 c0 82
(XEN) domain_crash called from realmode.c:168
(XEN) Domain 12 (vcpu#0) crashed on cpu#4:

(XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    4

(XEN) RIP:    f000:[<000000000000fff0>]

(XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest

(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000000000000
(XEN) rdx: 00000000078bfbff   rsi: 0000000000000000   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: 0000000000000000   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000030   cr4: 0000000000002040
(XEN) cr3: 0000000000000000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: f000
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22
(XEN) event_channel.c:270:d0 EVTCHNOP failure: domain 14, error -22

Thanks!
Etzion

On Wed, Dec 18, 2013 at 11:46 AM, Etzion Bar-Noy <etzion@xxxxxxxxxxxx> wrote:

Hi Mr. Yang.

I did attempt to use your suggested configuration with the XenServer VM, but to no avail. The L2 VM keeps on crushing. I defined it now to leave a dump file when crushing, and if you desire so, I will find some method of uploading it for your viewing pleasure. As I am not a programmer, but an integrator, my abilities in this direction are limited at best.

The L3 VM type is 'generic VM', because I wanted to enforce the system to use HVM emulation.

I have just applied the patch you supplied me to my 'staging' git checkout, and just finished compiling it. I will soon apply the newly created RPMs on my test (and hopefully - to be production soon) server, and will check VMware.

Thanks for the input. I will try and then respond with results.

Etzion

On Wed, Dec 18, 2013 at 3:33 AM, Zhang, Yang Z <yang.z.zhang@xxxxxxxxx> wrote:

Etzion Bar-Noy wrote on 2013-12-16:
> Hi.
>

Hi Etzion Bar-Noy

It's glad to know someone is using Nested Xen and report bugs to us. Also, if there are more nested bugs you found, please CC me too.

>
> I am using the unstable build-from-source of Xen community. It reports
> as 4.4, and was pulled from Git few days ago. The problem I will
> describe below appears on version 4.3.1, compiled from source as well.
> In both cases - I used spec file to create my own RPMs, because it's the right method of doing it.
>
>
> Problem: When VM running under HVM which is a hypervisor (XenServer
> 6.2 or VMware ESXi5.5) attempts to start a (HVM) virtual machine, the
> following happens, 100% reproducible on this server.
>
> XenServer: domain crash. XenServer restarts itself. It doesn't happen
> for PV on XenServer VM
>
> VMware: An "unknown failure", which leads me to think that the nested
> virtualization properties are not forwarded correctly to DomU

To using nested, you need to add the set bit 31 of cupid.1.ecx to 0, add the following to your config file:
cpuid = ['0x1:ecx=0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx']

For VMware ESX server, the attached patch is missing in current Xen which will caused the guest boot up fail. Also, I know some windows guests will boot fail on ESX.
I never tried XenServer. But latest Xen on Xen is working well. So can you try the upsteam Xen to see whether it fails too?

BTW, which L2 VM boot fail?

>
>
> Just to clarify:
>
> Physical hardware: host
>
> Nested hypervisor: level 1 virtualization: XenServer or VMware ESXi
>
> Nested VM: level 2 virtualization: some generic VM inside XenServer or
> ESXi (level 1)
>
>
> This is the configuration used for XS:
> name = "XenServer"
> uuid = "a0b8c668-afaf-887a-1f4b-856dac4c8401"
> maxmem = 4096
> memory = 4096
> vcpus = 2
> builder = "hvm"
> boot = "c"
> pae = 1
> acpi = 1
> apic = 1
> hap = 0
> viridian = 0
> rtc_timeoffset = 0
> localtime = 0
> > > > > > > sdl = 0
> vnc = 1
> vncunused = 1
> keymap = "en-us"
> disk = [ "tap2:aio:/dev/share/VMS/xenserver01.lun,hda,w",
> ",hdc:cdrom,r" ] vif = [
> "mac=00:16:3e:14:0d:72,bridge=lan,script=vif-bridge,model=e1000" ] parallel = "none"
> serial = "pty"
> nestedhvm = 1
>
> nx=1
>
>
> # A note - this problem appears with and without the 'nx' flag.
>
> A snippet from strace to qemu process:
>
> poll([{fd=30, events=POLLIN|POLLERR|POLLHUP}, {fd=18,
> events=POLLIN|POLLERR|POLLHUP}, {fd=29,
> events=POLLIN|POLLERR|POLLHUP}, {fd=21,
> events=POLLIN|POLLERR|POLLHUP}, {fd=15,
> events=POLLIN|POLLERR|POLLHUP}, {fd=5, events=

> POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN}, {fd=11, POLLIN|POLLERR|events=POLLIN}, {fd=28, events=POLLIN}, {fd=6, POLLIN|POLLERR|events=POLLIN}], 10, 307) = ?
> ERESTART_RESTARTBLOCK (To POLLIN|POLLERR|be restarted) --- SIGHUP

> (Hangup) @ 0 (0) --- write(6, "\1\0\0\0\0\0\0\0", 8)         = 8
> rt_sigreturn(0x2)                       = -1 EINTR (Interrupted system
> call) gettimeofday({1386761621, 705266}, NULL) = 0
> clock_gettime(CLOCK_MONOTONIC, {1840, 693236049}) = 0
> clock_gettime(CLOCK_MONOTONIC, {1840, 693333096}) = 0
> gettimeofday({1386761621, 705546}, NULL) = 0 write(2, "qemu: terminating
> on signal 1", 29) = 29 write(2, " from pid 6933\n", 15)        = 15
>
>
> From the xl log for this domain:
> Waiting for domain XenServer (domid 5) to die [pid 6933] Domain 5 has
> shut down, reason code 3 0x3 Action for shutdown reason code 3 is
> restart Domain
> 5 needs to be cleaned up: destroying the domain Done. Rebooting now
> Waiting for domain XenServer (domid 6) to die [pid 6933]
>
>
> From 'xl dmesg' (XEN) <vm_launch_fail> error code 7 (XEN)
> domain_crash_sync called from vmcs.c:1293 (XEN) Domain 8 (vcpu#0)
> crashed on cpu#4: (XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not
> tainted ]---- (XEN) CPU:    4 (XEN) RIP:    0000:[<0000000000000000>]
> (XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest (XEN) rax:
> 0000000000000000   rbx: ffff830824f98000   rcx: ffff830824f9ff80 (XEN)
> rdx: ffff82d0801d0cf9   rsi: 0000000000000000   rdi: ffff82d0801de7fc
> (XEN) rbp: ffff82d080105ab1   rsp: 0000000000000000   r8:
> 0000000000000004 (XEN) r9:  ffff82d080105b20   r10: ffff830824f9ff70
> r11: 0000000000000000 (XEN) r12: ffff830824f9ff50   r13:
> ffff830654f9e000   r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43
> cr0: 0000000000000039   cr4: 0000000000002050 (XEN) cr3:
> 00000000feffa000   cr2: 0000000000000000 (XEN) ds: 0000   es: 0000   fs:
> 0000   gs: 0000   ss: 0000   cs: 0000 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader (d9) Detected
> Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, event channel 4 (d9)
> System requested SeaBIOS (d9) CPU speed is 3400 MHz (d9) Relocating
> guest memory for lowmem MMIO space disabled (XEN) irq.c:270: Dom9 PCI
> link 0 changed 0 -> 5 (d9) PCI-ISA link 0 routed to IRQ5 (XEN)
> irq.c:270: Dom9 PCI link 1 changed 0 -> 10 (d9) PCI-ISA link 1 routed
> to IRQ10 (XEN)
> irq.c:270: Dom9 PCI link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed
> to
> IRQ11 (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA
> link
> 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) pci dev 02:0
> INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in high memory;
> setting high_mem resource base to 10f800000 (d9) pci dev 03:0 bar 10
> size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 size 001000000:
> 0f2000008 (d9) pci dev 04:0 bar 30 size 000040000: 0f3000000 (d9) pci
> dev 04:0 bar 10 size 000020000: 0f3040000 (d9) pci dev 03:0 bar 30
> size
> 000010000: 0f3060000 (d9) pci dev 03:0 bar 14 size 000001000:
> 0f3070000
> (d9) pci dev 02:0 bar 10 size 000000100: 00000c001 (d9) pci dev 04:0
> bar
> 14 size 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010:
> 00000c141 (d9) Multiprocessor initialisation: (d9)  - CPU0 ... 46-bit
> phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9)  - CPU1 ...
> 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) Testing
> HVM environment: (d9)  - REP INSB across page boundaries ... passed
> (d9)
>  - GS base MSRs and SWAPGS ... passed (d9) Passed 2 of 2 tests (d9)
> Writing SMBIOS tables ... (d9) Loading SeaBIOS ... (d9) Creating MP
> tables ... (d9) Loading ACPI ... (d9) vm86 TSS at fc00a080 (d9) BIOS
> map: (d9)  10000-100d3: Scratch space (d9)  e0000-fffff: Main BIOS
> (d9)
> E820 table: (d9)  [00]: 00000000:00000000 - 00000000:000a0000: RAM
> (d9)
> HOLE: 00000000:000a0000 - 00000000:000e0000 (d9)  [01]:
> 00000000:000e0000 - 00000000:00100000: RESERVED (d9)  [02]:
> 00000000:00100000 - 00000000:f0000000: RAM (d9)  HOLE:
> 00000000:f0000000
> - 00000000:fc000000 (d9)  [03]: 00000000:fc000000 - 00000001:00000000:
> RESERVED (d9)  [04]: 00000001:00000000 - 00000001:0f800000: RAM (d9)
> Invoking SeaBIOS ... (d9) SeaBIOS (version
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen
> hypervisor signature at 40000000 (d9) xen: copy e820... (d9)
> Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU
> Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated
> Xen hypercall page at effff000 (d9) Detected Xen v4.4-unstable (d9)
> xen: copy BIOS tables... (d9) Copying SMBIOS entry point from
> 0x00010010 to 0x000f1020
> (d9) Copying MPTABLE from 0xfc001170/fc001180 to 0x000f0f20 (d9)
> Copying PIR from 0x00010030 to 0x000f0ea0 (d9) Copying ACPI RSDP from
> 0x000100b0 to 0x000f0e70 (d9) Using pmtimer, ioport 0xb008, freq 3579
> kHz (d9) Scan for VGA option rom (d9) WARNING! Found unaligned PCI rom
> (vd=1013:00b8)
> (d9) Running option rom at c000:0003 (XEN) stdvga.c:147:d9 entering
> stdvga and caching modes (d9) Turning on vga text mode console (d9)
> SeaBIOS (version rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm)
> (d9) Machine UUID a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0
> lpt ports
> (d9) Found 1 serial ports (d9) ATA controller 1 at 1f0/3f4/c140 (irq
> 14 dev 9) (d9) ATA controller 2 at 170/374/c148 (irq 15 dev 9) (d9) ata0-0:
> QEMU HARDDISK ATA-7 Hard-Disk (14336 MiBytes) (d9) Searching bootorder
> for: /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM
> ATAPI-4 DVD/CD] (d9) Searching bootorder for:
> /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All
> threads complete. (d9) Scan for option roms (d9) Running option rom at
> c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 (d9) pmm call
> arg1=1
> (d9) pmm call arg1=0 (d9) Searching bootorder for: /pci@i0cf8/*@4 (d9)
> (d9) Press F12 for boot menu. (d9) (d9) Searching bootorder for: HALT
> (d9) drive 0x000f0e20: PCHS=16383/16/63 translation=lba
> LCHS=1024/255/63
> s=29360128 (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9)
> Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9)   0:
> 0000000000000000 - 000000000009fc00 = 1 RAM (d9)   1: 000000000009fc00 -
> 00000000000a0000 = 2 RESERVED (d9)   2: 00000000000f0000 -
> 0000000000100000 = 2 RESERVED (d9)   3: 0000000000100000 -
> 00000000effff000 = 1 RAM (d9)   4: 00000000effff000 - 00000000f0000000 =
> 2 RESERVED (d9)   5: 00000000fc000000 - 0000000100000000 = 2 RESERVED
> (d9)   6: 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter
> handle_19: (d9)   NULL (d9) Booting from Hard Disk... (d9) Booting from
> 0000:7c00 [root@monster ~]# xl dmesg (XEN) <vm_launch_fail> error code
> 7
> (XEN) domain_crash_sync called from vmcs.c:1293 (XEN) Domain 8
> (vcpu#0) crashed on cpu#4: (XEN) ----[ Xen-4.4-unstable  x86_64  debug=y  Not
> tainted ]---- (XEN) CPU:    4 (XEN) RIP:    0000:[<0000000000000000>]
> (XEN) RFLAGS: 0000000000000002   CONTEXT: hvm guest (XEN) rax:
> 0000000000000000   rbx: ffff830824f98000   rcx: ffff830824f9ff80 (XEN)
> rdx: ffff82d0801d0cf9   rsi: 0000000000000000   rdi: ffff82d0801de7fc
> (XEN) rbp: ffff82d080105ab1   rsp: 0000000000000000   r8:
> 0000000000000004 (XEN) r9:  ffff82d080105b20   r10: ffff830824f9ff70
> r11: 0000000000000000 (XEN) r12: ffff830824f9ff50   r13:
> ffff830654f9e000   r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43
> cr0: 0000000000000039   cr4: 0000000000002050 (XEN) cr3:
> 00000000feffa000   cr2: 0000000000000000 (XEN) ds: 0000   es: 0000   fs:
> 0000   gs: 0000   ss: 0000   cs: 0000 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0
> EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader (d9) Detected
> Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, event channel 4 (d9)
> System requested SeaBIOS (d9) CPU speed is 3400 MHz (d9) Relocating
> guest memory for lowmem MMIO space disabled (XEN) irq.c:270: Dom9 PCI
> link 0 changed 0 -> 5 (d9) PCI-ISA link 0 routed to IRQ5 (XEN)
> irq.c:270: Dom9 PCI link 1 changed 0 -> 10 (d9) PCI-ISA link 1 routed
> to IRQ10 (XEN)
> irq.c:270: Dom9 PCI link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed
> to
> IRQ11 (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA
> link
> 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) pci dev 02:0
> INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in high memory;
> setting high_mem resource base to 10f800000 (d9) pci dev 03:0 bar 10
> size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 size 001000000:
> 0f2000008 (d9) pci dev 04:0 bar 30 size 000040000: 0f3000000 (d9) pci
> dev 04:0 bar 10 size 000020000: 0f3040000 (d9) pci dev 03:0 bar 30
> size
> 000010000: 0f3060000 (d9) pci dev 03:0 bar 14 size 000001000:
> 0f3070000
> (d9) pci dev 02:0 bar 10 size 000000100: 00000c001 (d9) pci dev 04:0
> bar
> 14 size 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010:
> 00000c141 (d9) Multiprocessor initialisation: (d9)  - CPU0 ... 46-bit
> phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9)  - CPU1 ...
> 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) Testing
> HVM environment: (d9)  - REP INSB across page boundaries ... passed
> (d9)
>  - GS base MSRs and SWAPGS ... passed (d9) Passed 2 of 2 tests (d9)
> Writing SMBIOS tables ... (d9) Loading SeaBIOS ... (d9) Creating MP
> tables ... (d9) Loading ACPI ... (d9) vm86 TSS at fc00a080 (d9) BIOS
> map: (d9)  10000-100d3: Scratch space (d9)  e0000-fffff: Main BIOS
> (d9)
> E820 table: (d9)  [00]: 00000000:00000000 - 00000000:000a0000: RAM
> (d9)
> HOLE: 00000000:000a0000 - 00000000:000e0000 (d9)  [01]:
> 00000000:000e0000 - 00000000:00100000: RESERVED (d9)  [02]:
> 00000000:00100000 - 00000000:f0000000: RAM (d9)  HOLE:
> 00000000:f0000000
> - 00000000:fc000000 (d9)  [03]: 00000000:fc000000 - 00000001:00000000:
> RESERVED (d9)  [04]: 00000001:00000000 - 00000001:0f800000: RAM (d9)
> Invoking SeaBIOS ... (d9) SeaBIOS (version
> rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen
> hypervisor signature at 40000000 (d9) xen: copy e820... (d9)
> Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU
> Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated
> Xen hypercall page at effff000 (d9) Detected Xen v4.4-unstable (d9)
> xen: copy BIOS tables... (d9) Copying SMBIOS entry point from
> 0x00010010 to 0x000f1020
> (d9) Copying MPTABLE from 0xfc001170/fc001180 to 0x000f0f20 (d9)
> Copying PIR from 0x00010030 to 0x000f0ea0 (d9) Copying ACPI RSDP from
> 0x000100b0 to 0x000f0e70 (d9) Using pmtimer, ioport 0xb008, freq 3579
> kHz (d9) Scan for VGA option rom (d9) WARNING! Found unaligned PCI rom
> (vd=1013:00b8)
> (d9) Running option rom at c000:0003 (XEN) stdvga.c:147:d9 entering
> stdvga and caching modes (d9) Turning on vga text mode console (d9)
> SeaBIOS (version rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm)
> (d9) Machine UUID a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0
> lpt ports
> (d9) Found 1 serial ports (d9) ATA controller 1 at 1f0/3f4/c140 (irq
> 14 dev 9) (d9) ATA controller 2 at 170/374/c148 (irq 15 dev 9) (d9) ata0-0:
> QEMU HARDDISK ATA-7 Hard-Disk (14336 MiBytes) (d9) Searching bootorder
> for: /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM
> ATAPI-4 DVD/CD] (d9) Searching bootorder for:
> /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All
> threads complete. (d9) Scan for option roms (d9) Running option rom at
> c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 (d9) pmm call
> arg1=1
> (d9) pmm call arg1=0 (d9) Searching bootorder for: /pci@i0cf8/*@4 (d9)
> (d9) Press F12 for boot menu. (d9) (d9) Searching bootorder for: HALT
> (d9) drive 0x000f0e20: PCHS=16383/16/63 translation=lba
> LCHS=1024/255/63
> s=29360128 (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9)
> Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9)   0:
> 0000000000000000 - 000000000009fc00 = 1 RAM (d9)   1: 000000000009fc00 -
> 00000000000a0000 = 2 RESERVED (d9)   2: 00000000000f0000 -
> 0000000000100000 = 2 RESERVED (d9)   3: 0000000000100000 -
> 00000000effff000 = 1 RAM (d9)   4: 00000000effff000 - 00000000f0000000 =
> 2 RESERVED (d9)   5: 00000000fc000000 - 0000000100000000 = 2 RESERVED
> (d9)   6: 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter
> handle_19: (d9)   NULL (d9) Booting from Hard Disk... (d9) Booting from
> 0000:7c00
>
>
> System description:
>
> 1. Centos 6.4 x86_64, latest updates
>
> 2. Xen4centos packages - current kernel is
> 3.10.23-11.el6.centos.alt.x86_64
>
> 3. System with 32GB RAM
>
> 4. Virtual disks are ZFS volumes (aka - block devices). Same problem
> appeared when on local ext3 disks
>
> 5. Using OpenVSwitch version 2.0, compiled to RPM, again.
>
> 6. Intel CPU:
> processor    : 0
> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 62
> model name    : Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
> stepping    : 4
> microcode    : 0x416
> cpu MHz        : 3400.158
> cache size    : 12288 KB
> physical id    : 0
> siblings    : 12
> core id        : 0
> cpu cores    : 6
> apicid        : 0
> initial apicid    : 0
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 13
> wp        : yes
> flags        : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi
> mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl
> nonstop_tsc eagerfpu pni pclmulqdq monitor est ssse3 cx16 sse4_1
> sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor
> lahf_lm ida arat epb xsaveopt pln pts dtherm fsgsbase erms
> bogomips    : 6800.31
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 46 bits physical, 48 bits virtual
> power management:
>
>
> It was suggested by Mr. Ian Campbell to CC both Mr. Jun Nakajima and Mr.
> Eddie Dong, so, my apologies if this action is improper.
>
>
> Any tip or advice would be great. Any request for further info would
> be also great!
>
> Thanks!
>
> Etzion Bar-Noy

Best regards,
Yang

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel