[Xen-devel] FreeBSD PVH guest support

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

Hello,

The Xen community is working on a new virtualization mode (or maybe I 
should say an extension of HVM) to be able to run PV guests inside HVM 
containers without requiring a device-model (Qemu). One of the 
advantages of this new virtualization mode is that now it is much more 
easier to port guests to run under it (as compared to pure PV guests).

Given that FreeBSD already supports PVHVM, adding PVH support is quite 
easy, we only need some glue for the PV entry point and then support 
for diverging some early init functions (like fetching the e820 map or 
starting the APs).

The attached patch contains all this changes, and allows a SMP FreeBSD 
guest to fully boot (and AFAIK work) under this new PVH mode. The patch 
can also be found on my git repo:

git://xenbits.xen.org/people/royger/freebsd.git pvh_v2

The patch touches quite a lot of the early init, so I've Cced the 
persons that maintain those areas, so they can review it.

In order to test it, and since the PVH changes are not yet merged into 
upstream Xen, the use of a patched Xen is necessary. I've collected the 
patches for PVH guest support from George Dunlap (v13) and fixed some 
bugs on top of them, the tree can be found at:

git://xenbits.xen.org/people/royger/xen.git fix_pvh

For those curious, here is a dmesg of a FreeBSD PVH guest booting:

GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
SMAP type=01 base=0000000000000000 len=0000000138800000
ACPI BIOS Error (bug): A valid RSDP was not found (20130823/tbxfroot-223)
APIC: Using the Xen PV enumerator.
SMP: Added CPU 0 (BSP)
SMP: Added CPU 2 (AP)
SMP: Added CPU 4 (AP)
SMP: Added CPU 6 (AP)
SMP: Added CPU 8 (AP)
SMP: Added CPU 10 (AP)
SMP: Added CPU 12 (AP)
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #420: Mon Oct 28 13:07:53 CET 2013
    root@odin:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
WARNING: WITNESS option enabled, expect reduced performance.
Hypervisor: Origin = "XenVMMXenVMM"
Calibrating TSC clock ... TSC clock: 3066775691 Hz
CPU: Intel(R) Xeon(R) CPU           W3550  @ 3.07GHz (3066.78-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106a5  Family = 0x6  Model = 0x1a  Stepping = 
5
  
Features=0x1fc98b75<FPU,DE,TSC,MSR,PAE,CX8,APIC,SEP,CMOV,PAT,CLFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0x80982201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,HV>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
real memory  = 5242880000 (5000 MB)
Physical memory chunk(s):
0x0000000000010000 - 0x00000000001fffff, 2031616 bytes (496 pages)
0x0000000002708000 - 0x0000000130864fff, 5068148736 bytes (1237341 pages)
avail memory = 5035581440 (4802 MB)
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 4 as a target
INTR: Adding local APIC 6 as a target
INTR: Adding local APIC 8 as a target
INTR: Adding local APIC 10 as a target
INTR: Adding local APIC 12 as a target
FreeBSD/SMP: Multiprocessor System Detected: 7 CPUs
FreeBSD/SMP: 1 package(s) x 7 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  2
 cpu2 (AP): APIC ID:  4
 cpu3 (AP): APIC ID:  6
 cpu4 (AP): APIC ID:  8
 cpu5 (AP): APIC ID: 10
 cpu6 (AP): APIC ID: 12
XEN: CPU 0 has VCPU ID 0
XEN: CPU 1 has VCPU ID 1
XEN: CPU 2 has VCPU ID 2
XEN: CPU 3 has VCPU ID 3
XEN: CPU 4 has VCPU ID 4
XEN: CPU 5 has VCPU ID 5
XEN: CPU 6 has VCPU ID 6
x86bios:  IVT 0x000000-0x0004ff at 0xfffff80000000000
x86bios: SSEG 0x010000-0x010fff at 0xfffffe012e79d000
x86bios:  ROM 0x0a0000-0x0fefff at 0xfffff800000a0000
random device not loaded; using insecure entropy
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
Event-channel device installed.
snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 
feeder_rate_max=2016000 feeder_rate_round=25
wlan: <802.11 Link Layer>
Hardware, VIA Nehemiah Padlock RNG: VIA Padlock RNG not present
Hardware, Intel IvyBridge+ RNG: RDRAND is not present
null: <null device, zero device>
Falling back to <Software, Yarrow> random adaptor
random: <Software, Yarrow> initialized
nfslock: pseudo-device
kbd0 at kbdmux0
module_register_init: MOD_LOAD (vesa, 0xffffffff80d21c60, 0) error 19
io: <I/O>
VMBUS: load
mem: <memory>
hpt27xx: RocketRAID 27xx controller driver v1.1
hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2
hptnr: R750/DC7280 controller driver v1.0
ACPI BIOS Error (bug): A valid RSDP was not found (20130823/tbxfroot-223)
ACPI: Table initialisation failed: AE_NOT_FOUND
ACPI: Try disabling either ACPI or apic support.
xenstore0: <XenStore> on motherboard
Grant table initialized
xc0: <Xen Console> on motherboard
xen_et0: <Xen PV Clock> on motherboard
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
xen_et0: registered as a time-of-day clock (resolution 10000000us, adjustment 
5.000000000s)
pvcpu0: <Xen PV CPU> on motherboard
pvcpu1: <Xen PV CPU> on motherboard
pvcpu2: <Xen PV CPU> on motherboard
pvcpu3: <Xen PV CPU> on motherboard
pvcpu4: <Xen PV CPU> on motherboard
pvcpu5: <Xen PV CPU> on motherboard
pvcpu6: <Xen PV CPU> on motherboard
legacy_pcib_identify: no bridge found, adding pcib0 anyway
pcib0 pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
pci0: domain=0, physical bus=0
cpu0 on motherboard
cpu1 on motherboard
cpu2 on motherboard
cpu3 on motherboard
cpu4 on motherboard
cpu5 on motherboard
cpu6 on motherboard
isa0: <ISA bus> on motherboard
qpi0: <QPI system bus> on motherboard
ex_isa_identify()
isa_probe_children: disabling PnP devices
isa_probe_children: probing non-PnP devices
fb: new array size 4
sc0: <System console> on isa0
sc0: MDA <16 virtual consoles, flags=0x100>
sc0: fb0, kbd0, terminal emulator: scteken (teken terminal)
vga0: <Generic ISA VGA> at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0
isa_probe_children: probing PnP devices
Device configuration finished.
procfs registered
Timecounters tick every 1.000 msec
vlan: initialized, using hash tables with chaining
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 65536
lo0: bpf attached
hpt27xx: no controller detected.
hptrr: no controller detected.
hptnr: no controller detected.
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: bpf attached
xn0: Ethernet address: 00:16:3e:0b:a4:b1
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xbd0: 20480MB <Virtual Block Device> at device/vbd/51712 on xenbusb_front0
xbd0: features: flush, write_barrier
xbd0: synchronize cache commands enabled.
GEOM: new disk xbd0
random: unblocking device.
Netvsc initializing... SMP: AP CPU #5 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #4 Launched!
TSC timecounter discards lower 1 bit(s)
Timecounter "TSC-low" frequency 1533387845 Hz quality -100
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/xbd0p2 []...
start_init: trying /sbin/init
Setting hostuuid: c9230f36-1a54-489e-877c-1d15b8f463e9.
Setting hostid: 0xd52252c7.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No such 
file or directory
 kickstart.
Starting file system checks:
/dev/xbd0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/xbd0p2: clean, 2213647 free (17111 frags, 274567 blocks, 0.4% 
fragmentation)
Mounting local file systems:.
Writing entropy file:.
xn0: link state changed to DOWN
xn0: link state changed to UP
Starting Network: lo0 xn0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=503<RXCSUM,TXCSUM,TSO4,LRO>
        ether 00:16:3e:0b:a4:b1
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet manual
        status: active
Starting devd.
Starting dhclient.
DHCPDISCOVER on xn0 to 255.255.255.255 port 67 interval 7
DHCPOFFER from 172.16.1.1
DHCPREQUEST on xn0 to 255.255.255.255 port 67
DHCPACK from 172.16.1.1
bound to 172.16.1.149 -- renewal in 43200 seconds.
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
32-bit compatibility ldconfig path: /usr/lib32
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
lock order reversal:
 1st 0xfffffe012e861e28 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3050
 2nd 0xfffff80005b87c00 dirhash (dirhash) @ 
/usr/src/sys/ufs/ufs/ufs_dirhash.c:284
KDB: stack backtrace:
X_db_symbol_values() at X_db_symbol_values+0x10b/frame 0xfffffe012fb8c410
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe012fb8c4c0
witness_checkorder() at witness_checkorder+0xd23/frame 0xfffffe012fb8c550
_sx_xlock() at _sx_xlock+0x75/frame 0xfffffe012fb8c590
ufsdirhash_add() at ufsdirhash_add+0x3b/frame 0xfffffe012fb8c5d0
ufs_direnter() at ufs_direnter+0x688/frame 0xfffffe012fb8c690
ufs_vinit() at ufs_vinit+0x33f3/frame 0xfffffe012fb8c890
VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xfffffe012fb8c8c0
kern_mkdirat() at kern_mkdirat+0x1ff/frame 0xfffffe012fb8cae0
amd64_syscall() at amd64_syscall+0x265/frame 0xfffffe012fb8cbf0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe012fb8cbf0
--- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80092faaa, rsp = 
0x7fffffffd788, rbp = 0x7fffffffdc70 ---
Clearing /tmp (X related).
Updating motd:.
Configuring syscons: keymap blanktime.
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Starting background file system checks in 60 seconds.

Mon Oct 28 13:22:52 CET 2013

FreeBSD/amd64 (Amnesiac) (xc0)

Attachment: 0001-Xen-x86-PVH-support.patch
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel