Re: [Xen-users] DomU not starting under pvhv2

[Christopher Myers <cmyers@xxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512



On Mon, 2018-01-08 at 22:01 +1300, Peter wrote:
> Hi.
> 
> Running Xen 4.10.0
> 
> A VM is not starting with type = 'pvh'.  The VM starts, but exits prior 
> to any data being read off the domU disk image.
> 
> The same VMs works if the type is changed to 'pv'.
> 
> Other VMs on other hosts (with the exact same setup) work with pvh fine.
> 
> The main difference between hosts where type=pvh does not work and ones 
> where it does is that the pvh does not work on older CPUs:
>   Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           E3110  @ 3.00GHz
>   Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz
>   Intel(R) Xeon(R) CPU           E5320  @ 1.86GHz
>   Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
>   Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz
> 
> 
> Newer CPUs seem to be fine (start VMs with type = pvh):
>   Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
>   Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
>   Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
>   Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
>   Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
>   Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>   Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>   Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
>   Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
>   Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
>   Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
>   Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
>   Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
> 
> Dom0 command line is: dom0_max_vcpus=2 dom0_vcpus_pin loglvl=info 
> vcpu_migration_delay=1000
> 
> I see (all?) the older CPUs don't have VT-d.  However some of the CPUs 
> where pvh is working do not report hvm_directio in xl info virt_caps 
> (e.g. the E5506 servers) though they do have VT-d available per 
> ark.intel.com.


Out of curiosity, just because the CPU supports VT-d doesn't mean that
everything else in the system does or that it's enabled in the BIOS. My
work computer, for example, supports it, but I had to explicitly enable
it first. So just something to double-check.


> 
And ideas on what may be stopping type=pvh VMs starting on those hosts?  
> And how we could get it working?
> 
> Else we'll likely be dumping some hardware....
> 
> I did a diff of xl info; xl dmesg and xl create startup messages.  
> Between one of the non-pvh-able hosts (the - line) and a host where pvh 
> works (the + line):
> diff host of xl -v create and xlinfo:
> -hw_caps                : 
> bfebfbff:0004e3bd:20100800:00000001:00000000:00000000:00000000:00000000
> -virt_caps              : hvm
> +hw_caps                : 
> bfebfbff:009ee3fd:2c100800:00000001:00000000:00000000:00000000:00000100
> +virt_caps              : hvm hvm_directio
> 
> +libxl: detail: libxl_dom.c:204:numa_place_domain: NUMA placement 
> candidate with 1 nodes, 8 cpus and 16819 KB free selected
> 
> - (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
> + (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
> 
> - (XEN) I/O virtualisation disabled
> + (XEN) Intel VT-d Snoop Control enabled.
> + (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> + (XEN) Intel VT-d Queued Invalidation enabled.
> + (XEN) Intel VT-d Interrupt Remapping enabled.
> + (XEN) Intel VT-d Posted Interrupt not enabled.
> + (XEN) Intel VT-d Shared EPT tables not enabled.
> + (XEN) I/O virtualisation enabled
> 
> -(XEN)  -> Using new ACK method
> +(XEN)  -> Using old ACK method
> 
>    (XEN) VMX: Supported advanced features:
> + (XEN)  - APIC MMIO access virtualisation
> + (XEN)  - Extended Page Tables (EPT)
> + (XEN)  - Virtual-Processor Identifiers (VPID)
> + (XEN)  - Virtual NMI
> + (XEN)  - Unrestricted Guest
> 
> 
> -(XEN) HVM: Hardware Assisted Paging (HAP) not detected
> +(XEN) HVM: Hardware Assisted Paging (HAP) detected
> 
> +(XEN) Found masked UR signaling on 0000:00:00.0
> +(XEN) Masked UR signaling on 0000:00:01.0
> +(XEN) Masked UR signaling on 0000:00:03.0
> +(XEN) Masked UR signaling on 0000:00:05.0
> +(XEN) Masked UR signaling on 0000:00:07.0
> +(XEN) Masked VT-d error signaling on 0000:00:14.0
> 
> 
> Regards, Peter
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxxx
> 


> https://lists.xenproject.org/mailman/listinfo/xen-users
-----BEGIN PGP SIGNATURE-----

iQEyBAEBCgAdFiEE7GM/Dul8WSWn72odQ1nEo4DFCIUFAlpTkEsACgkQQ1nEo4DF
CIWHCQf4mB1U1V+J0j1C1Dr7Zz3QG7dc1rb8iAKEjDQCTMP4I5vke9jf7fDUNFaF
JI3C3SEzY1XeedRsywedqlkZKkk+RK0W5B0rUkpEGbjopGj4VuIB7G8qntWUnTHT
5kUmPAtQwjRHPypZIHFK7uhqI/3WgsnUy85du76q+3MzXE4NFx+hQ0wMbhCWrcyK
pUoWh+dFybsJPRAEGHv4FSUwVOdzNDocKbqPDd11wLG/Lw3tqguRCbHUhOwycUlP
vIKD21EtZ59Iq5oyqoGkIZQMIKA7JRrtSMvmwHSS2gp4p+4tYpK32A9UWIJj1jZD
ieX6l7d1cwnxp89D2MwtIYf7pdnj
=RvxY
-----END PGP SIGNATURE-----
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users