Re: [Xen-API] XCP 1.1 xs-tools Version Override Problem

[brooks@xxxxxxxxxxx]

On Sat, 22 Oct 2011, brooks@xxxxxxxxxxx wrote:

> I suspect that didn't get done, or isn't working.  What functions do the 
> xe-guest-utilities provide?
I thought I'd answer my own question so everyone can benefit.  I ripped 
into the sources to get to the bottom of the xs-tools mystery.  In general 
it provides the following:
    1. xs-tools sets up a local xenstore cache on the VM installs a bunch
       of utilities to manage (add, delete, and destroy) the cache.

    2. Provides a (broken as far as I can tell) CPU hotplug hack.  This
       may have been, or may still be useful for some distros.

    3. Determines the OS type and version and adds that information to
       xenstore.

    4. Adds xs-tools version db entries to xenstore.

    5. Sets up VM memory usage monitoring.

    6. Enables memory ballooning.

    7. Installs a xenified kernel if needed (not need for modern distros).

xs-tools has always been a problem in terms of VM management.  Ops would 
like to be able to upgrade XenServer/XCP and not have to get involved with 
end user VMs but xs-tools makes that impossible.  As things are 
architected today an upgrading XenServer or XCP also requires an upgrade 
to xs-tools on ever VM in your pool.  That process requires mounting the 
ISO and running some flavor of install.sh, or installing the RPMs 
directly, depending on your level of knowledge and experience.  Ensuring 
that all of your installed VMs are updated when you may not have root 
access to the VM is extremely difficult and (IMO) one of the biggest 
issues with managing XenServer/XCP vs. Xen or KVM.
Since CloudStack manages XenServer/XCP I wondered how they worked around 
the xs-tools problem.  In the current version (2.2.12) they use the George 
Shuklin hack:
<http://lists.xensource.com/archives/html/xen-api/2011-03/msg00031.html>

to fake XenServer/XCP into thinking the correct version of xs-tools is 
installed.  In fact the function they use to set the xenstore values is 
called "fake".  CloudStack docs still state that you need to install the 
latest version of xs-tools on your domU VMs but I'm not sure the docs are 
entirely in sync with the code.  When you instantiate a new VM they call a 
script installed into the compute node to set the PV-drivers-version 
values to match what they find in /etc/xensource-inventory.
fake(){
  local uuid=$1
  domid=$(xe vm-list uuid=${uuid} params=dom-id --minimal)
  xenstore-write /local/domain/${domid}/attr/PVAddons/MajorVersion ${major} \
                 /local/domain/${domid}/attr/PVAddons/MinorVersion ${minor} \
                 /local/domain/${domid}/attr/PVAddons/MicroVersion ${micro} \
                 /local/domain/${domid}/data/updated 1
}

In terms of CloudStack we could work with the xapi version override hack 
available in XCP if instead of getting the installed version of 
XenServer/XCP from /etc/xensource-inventory it instead made a call using 
xe.  For example:
[root@xenc2n2 ~]# xe host-list params=software-version 
uuid=d11bd238-0ab5-4b2d-8e33-68b090aaf6e1
software-version (MRO)    : product_version: 5.6.100; 
product_version_text: 5.6.100; product_version_text_short: 5.6.100; 
product_brand: XCP; build_number: 50674c; hg_id: ; hostname: kiffu-2; 
date: 2011-09-19; dbv: 2009.0201; xapi: 1.3; xen: 3.4.2; linux: 
2.6.32.12-0.7.1.xs1.1.0.327.170596xen; xencenter_min: 1.8; xencenter_max: 
1.8; network_backend: openvswitch; xs:main: Base Pack, version 1.1.0, 
build 50674c
Given what xs-tools actually provides to current PV based Linux distros I 
think the "George S. pv-drivers version fake hack" is all you really need. 
What I don't know is what CloudStack does when you upgrade the pool (they 
call it a cluster) from Xen X to Xen Y.  But there's no reason they 
couldn't run their make_migratable.sh script on all of the existing VMs as 
part of the upgrade process.  The real beauty of that solution is that 
there's no end user interaction needed to upgrade your pool and all of 
your VMs to the "latest" version of xs-tools, an essential requirement for 
using XenServer/XCP in a muti-tennent environment.
The one nice feature that xs-tools does provide is VM memory usage 
monitoring.  If you really wanted it, as it does give you a nice view into 
your VM resource usage from XenCenter you could easily enable it yourself. 
Memory Ballooning isn't currently supported by CloudStack, but again if 
you wanted to enable it for use in a XenCenter based environment it's as 
simple as making one addition to the xenstore db.
    xenstore_write "control/feature-balloon" "1"

The question I have is whether not using the xenstore local cache will 
have an adverse affect on the performance of XAPI/xenstore db?  Is this 
why CloudStack "Best Practices" state that you should limit your 
XenServer/XCP pools to 8 or less compute nodes?

_______________________________________________
xen-api mailing list
xen-api@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/mailman/listinfo/xen-api