[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-API] XCP 1.1 xs-tools Version Override Problem

On Sat, 22 Oct 2011, brooks@xxxxxxxxxxx wrote:

I suspect that didn't get done, or isn't working. What functions do the xe-guest-utilities provide?

I thought I'd answer my own question so everyone can benefit. I ripped into the sources to get to the bottom of the xs-tools mystery. In general it provides the following:

    1. xs-tools sets up a local xenstore cache on the VM installs a bunch
       of utilities to manage (add, delete, and destroy) the cache.

    2. Provides a (broken as far as I can tell) CPU hotplug hack.  This
       may have been, or may still be useful for some distros.

    3. Determines the OS type and version and adds that information to

    4. Adds xs-tools version db entries to xenstore.

    5. Sets up VM memory usage monitoring.

    6. Enables memory ballooning.

    7. Installs a xenified kernel if needed (not need for modern distros).

xs-tools has always been a problem in terms of VM management. Ops would like to be able to upgrade XenServer/XCP and not have to get involved with end user VMs but xs-tools makes that impossible. As things are architected today an upgrading XenServer or XCP also requires an upgrade to xs-tools on ever VM in your pool. That process requires mounting the ISO and running some flavor of install.sh, or installing the RPMs directly, depending on your level of knowledge and experience. Ensuring that all of your installed VMs are updated when you may not have root access to the VM is extremely difficult and (IMO) one of the biggest issues with managing XenServer/XCP vs. Xen or KVM.

Since CloudStack manages XenServer/XCP I wondered how they worked around the xs-tools problem. In the current version (2.2.12) they use the George Shuklin hack:


to fake XenServer/XCP into thinking the correct version of xs-tools is installed. In fact the function they use to set the xenstore values is called "fake". CloudStack docs still state that you need to install the latest version of xs-tools on your domU VMs but I'm not sure the docs are entirely in sync with the code. When you instantiate a new VM they call a script installed into the compute node to set the PV-drivers-version values to match what they find in /etc/xensource-inventory.

  local uuid=$1
  domid=$(xe vm-list uuid=${uuid} params=dom-id --minimal)
  xenstore-write /local/domain/${domid}/attr/PVAddons/MajorVersion ${major} \
                 /local/domain/${domid}/attr/PVAddons/MinorVersion ${minor} \
                 /local/domain/${domid}/attr/PVAddons/MicroVersion ${micro} \
                 /local/domain/${domid}/data/updated 1

In terms of CloudStack we could work with the xapi version override hack available in XCP if instead of getting the installed version of XenServer/XCP from /etc/xensource-inventory it instead made a call using xe. For example:

[root@xenc2n2 ~]# xe host-list params=software-version uuid=d11bd238-0ab5-4b2d-8e33-68b090aaf6e1 software-version (MRO) : product_version: 5.6.100; product_version_text: 5.6.100; product_version_text_short: 5.6.100; product_brand: XCP; build_number: 50674c; hg_id: ; hostname: kiffu-2; date: 2011-09-19; dbv: 2009.0201; xapi: 1.3; xen: 3.4.2; linux:; xencenter_min: 1.8; xencenter_max: 1.8; network_backend: openvswitch; xs:main: Base Pack, version 1.1.0, build 50674c

Given what xs-tools actually provides to current PV based Linux distros I think the "George S. pv-drivers version fake hack" is all you really need. What I don't know is what CloudStack does when you upgrade the pool (they call it a cluster) from Xen X to Xen Y. But there's no reason they couldn't run their make_migratable.sh script on all of the existing VMs as part of the upgrade process. The real beauty of that solution is that there's no end user interaction needed to upgrade your pool and all of your VMs to the "latest" version of xs-tools, an essential requirement for using XenServer/XCP in a muti-tennent environment.

The one nice feature that xs-tools does provide is VM memory usage monitoring. If you really wanted it, as it does give you a nice view into your VM resource usage from XenCenter you could easily enable it yourself. Memory Ballooning isn't currently supported by CloudStack, but again if you wanted to enable it for use in a XenCenter based environment it's as simple as making one addition to the xenstore db.

    xenstore_write "control/feature-balloon" "1"

The question I have is whether not using the xenstore local cache will have an adverse affect on the performance of XAPI/xenstore db? Is this why CloudStack "Best Practices" state that you should limit your XenServer/XCP pools to 8 or less compute nodes?

xen-api mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.