Re: [Xen-users] Wrong version of xen-utils after building xen from source causes xen to exit with error

[Brett Stahlman <brettstahlman@xxxxxxxxx>]

** Update **
Following recommendations on a page dealing with the installation of
multiple versions of Xen side-by-side, I re-ran configure like this...

configure --prefix=/home/bstahlman/install/xen \
--sysconfdir=/home/bstahlman/install/xen/etc \
--enable-rpath

...and was then able to run xl by specifying the path to the sbin/xl
under the directory specified with --prefix.

Aside: I also had to add this line to /etc/fstab...

none /proc/xen xenfs defaults 0 0

...and I have to start the xencommons service manually like so:

sudo ~/install/xen/etc/init.d/xencommons start

But now I'm able to run xen-create-image to create an ubuntu guest:
sudo xen-create-image --hostname ubuntu-test --dhcp --dir
~/xen/images/ --size=4G --role=udev

(Note that I had to use the script installed by package manager, as I
don't see a version of xen-create-image under my custom install
directory). Strangely, xen-create-image gives what looks like an
error...

ERROR: Can't find version 4.8 of xen utils, bailing out!

...but the command appears to work, and I'm able to start the guest by
running xl on the resulting config file:
sudo ./install/xen/sbin/xl create /etc/xen/ubuntu-test.cfg -c

The problem occurs when I attempt to create an HVM guest:
$ sudo ~/install/xen/sbin/xl create ~/cfg/xen/xl-ubuntu-hvmloader.cfg -c
Parsing config from /home/bstahlman/cfg/xen/xl-ubuntu-hvmloader.cfg
libxl: error: libxl_create.c:562:libxl__domain_make: domain creation
fail: Invalid argument
libxl: error: libxl_create.c:904:initiate_domain_create: cannot make domain: -3

I initially assumed the error must be due to the options in the config
file (pasted at the end), but when I ran the xl create command in gdb,
I discovered that the option parsing was successful: the error occurs
because the ioctl() call in the following function returns -1:

int osdep_hypercall(xencall_handle *xcall, privcmd_hypercall_t *hypercall)
{
    return ioctl(xcall->fd, IOCTL_PRIVCMD_HYPERCALL, hypercall);
}

Here's the backtrace showing the context:
#0  osdep_hypercall (xcall=0x639770, hypercall=0x7fffffffdf90) at linux.c:68
#1  0x00007ffff5c6002e in xencall1 (xcall=0x639770, op=36,
arg1=140737354088452) at core.c:92
#2  0x00007ffff6ac9af8 in do_domctl (xch=0x639310,
domctl=0x7fffffffe0b0) at xc_private.h:273
#3  0x00007ffff6ac9cf2 in xc_domain_create (xch=0x639310, ssidref=11,
    handle=0x7fffffffe200
"V\304\325\377\303tEX\201\062\224\347}\341\375-0\342\377\377\377\177",
flags=3, pdomid=0x7fffffffe260, config=0x63b584)
    at xc_domain.c:61
#4  0x00007ffff791ea40 in libxl__domain_make (gc=0x63a990,
d_config=0x7fffffffe570, domid=0x7fffffffe260, xc_config=0x63b584) at
libxl_create.c:559
#5  0x00007ffff791fdbe in initiate_domain_create (egc=0x7fffffffe350,
dcs=0x63b060) at libxl_create.c:902
#6  0x00007ffff79223b6 in do_domain_create (ctx=0x639050,
d_config=0x7fffffffe570, domid=0x7fffffffe42c, restore_fd=-1,
send_back_fd=-1, params=0x0,
    colo_proxy_script=0x0, ao_how=0x0, aop_console_how=0x7fffffffe500)
at libxl_create.c:1627
#7  0x00007ffff7922f08 in libxl_domain_create_new (ctx=0x639050,
d_config=0x7fffffffe570, domid=0x7fffffffe42c, ao_how=0x0,
aop_console_how=0x7fffffffe500)
    at libxl_create.c:1811
#8  0x0000000000412f82 in create_domain (dom_info=0x7fffffffea20) at
xl_cmdimpl.c:3047
#9  0x000000000041a323 in main_create (argc=2, argv=0x7fffffffebc8) at
xl_cmdimpl.c:5530
#10 0x0000000000408e8a in main (argc=3, argv=0x7fffffffebc0) at xl.c:364

Any ideas on how I should debug this? I believe the error code
returned by ioctl is EPERM, which isn't listed as one of the possible
return values in the man page. Below is the xl.cfg file I'm using for
the HVM guest. Note that I've tried setting device_model_version to
both 'qemu-xen' and 'qemu-xen-traditional'.

Thanks,
Brett S.

# =====================================================================
# Example HVM guest configuration
# =====================================================================
#
# This is a fairly minimal example of what is required for an
# HVM guest. For a more complete guide see xl.cfg(5)

boot = "dc"

device_model_version = 'qemu-xen-traditional'

# This configures an HVM rather than PV guest
builder = "hvm"

# Guest name
name = "ubuntu-hvmloader"

# 128-bit UUID for the domain as a hexadecimal number.
# Use "uuidgen" to generate one if required.
# The default behavior is to generate a new UUID each time the guest is started.
#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

# Enable Microsoft Hyper-V compatibile paravirtualisation /
# enlightenment interfaces. Turning this on can improve Windows guest
# performance and is therefore recommended
#viridian = 1

# Initial memory allocation (MB)
memory = 1000

# Maximum memory (MB)
# If this is greater than `memory' then the slack will start ballooned
# (this assumes guest kernel support for ballooning)
#maxmem = 512

# Number of VCPUS
vcpus = 2

# Network devices
# A list of 'vifspec' entries as described in
# docs/misc/xl-network-configuration.markdown
vif = [ '' ]

# Disk Devices
# A list of `diskspec' entries as described in
# docs/misc/xl-disk-configuration.txt
disk = [
'/home/bstahlman/xen/images/domains/ubuntu-hvmloader/disk.img,raw,hda,rw',
'/home/bstahlman/Downloads/ubuntu-16.04.1-desktop-amd64.iso,,hdc,cdrom'
]

# Guest VGA console configuration, either SDL or VNC
sdl = 1
#vnc = 1



On Tue, Sep 6, 2016 at 11:35 AM, Brett Stahlman <brettstahlman@xxxxxxxxx> wrote:
> Hello,
> I was able to build the latest version of xen (4.8) from source with
> the following commands on Ubuntu 16.04 LTS:
>
> ./configure --enable-ovmf
> make world
> make install
>
> After the install, it appeared that the xen.gz in my /boot dir was the
> one I had just built, so I rebooted and chose the Ubuntu with Xen
> Hypervisor option, and the boot appeared at first to be successful.
> However, when I attempted to connect to Xen using virt-manager, I got
> an error along the lines of "libxenlight state driver is not active".
> The results of my search on this error led me to suspect that I didn't
> have the proper version of xen-utils installed. Invoking xen from the
> command line in an attempt to determine the xen version gave the
> following error...
>
> ERROR:  Can't find version 4.8 of xen utils, bailing out!
>
> ...which served to confirm my suspicions regarding xen-utils. But
> wouldn't a source build of xen itself also build the utils? I notice
> there's a "dist-tools" target: installing it with `make install
> dist-tools' gives no error, but doesn't seem to fix the problem. So
> I'm not sure whether the problem is that it's not being installed to a
> location where xen will find it, or whether dist-tools is something
> entirely different from the "xen-utils" package.
>
> Incidentally, when I attempt to install the xen-utils package using
> apt-get, I get dependency errors. Perhaps this is because the newer
> version of xen I've installed from source is somehow conflicting with
> the version the package manager wants to install (4.6)?
>
> So I guess my question boil down to the following...
> Does xen `make world' build what the package manager refers to as
> "xen-utils"? If so, how do I install it in such a way that xen will
> find it, rather than the older version installed by the package
> manager. (I thought about using the package manager to uninstall all
> xen-related packages, but this would remove a lot of libraries, and I
> was afraid of breaking something else that could depend on some of
> them.)
>
> Is there a different way I should be going about this? The reason I'm
> building xen from source is that I want to be able to insert print
> statements in the code that loads bios/uefi firmware blobs (e.g.,
> seabios, ovmf, etc...) to shed some light on the guest boot process...
> If I had a way to set up debugging on the stock installation, I might
> be able to dispense with the source build altogether, but I'm
> relatively new to Xen, and have no idea what sort of debugging is
> possible.
>
> Thanks,
> Brett S.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
https://lists.xen.org/xen-users