Re: [Xen-devel] [PATCH] xenctld - a control channel multiplexing daemon

[Steven Hand <Steven.Hand@xxxxxxxxxxxx>]

> On Thu, 2005-01-27 at 08:19 -0600, Anthony Liguori wrote:
> > Daniel Stekloff wrote:
> > 
> > >Can't we add functionality in Xen to make domain id's unique? When
> > >creating a new domain, the management tools can query the running (or
> > >even suspended) domains and find a unique domain id to use. I think you
> > >can tag a domains with their state. 
> > >
> > >  
> > >
> > Yes, I think eventually domains have to have some sort of UUID.  I'm not 
> > convinced that those can't be mapped onto the current domain ids by 
> > configuration tools though.
> > 
> > Here's a few scenarios to consider:
> > 
> > 1) Someone suspends domain A to disk.  Domain A continues to run.  
> > Without stopping the original instance of Domain A, they start up the 
> > suspended image.
> 
> At some point you need to make rules for operating, deciding based on
> how you plan to use it what is the right way and the wrong way. A domain
> needs to be unique. It can move through phases like being active or
> being suspended. In the case you mention, I don't think Domain A can be
> suspended and active at the same time. It's one or the other. If you
> need to restart from a checkpoint, you clone the Domain A to Domain B.
> We could think of it like process management.

So it may be useful to revisit one of the original distinctions that
we made, that between "domain" and "virtual machine": 

  - A domain is a Xen concept that has an associated domain struct,
    allocation of physical memory, CPU time, event channels, etc. A 
    domain has an ID which is unique to all the domains currently
    existing on a given physical machine. In general, think of "domain" 
    as the analog of "process".

  - A virtual machine is the thing that inhabits a domain. It may 
    inhabit different domains (and hence have different domain IDs) 
    at various points in its life-cycle (e.g. due to suspend/resume, 
    live migration or even reboot). 

Most management tools will want to manage virtual machines as first
class entities, and worry less about domains (all that is required 
is a dynamic mapping from VM to (machine id, domain id)). In the 
current tools (xend etc), VMs are named by system-wide ASCII names. 
Xen knows (and cares) nothing about these. The tools make a basic 
effort to ensure that VM names are unique but currently this is 
only guaranteed per physical machine rather than per cluster; furhermore 
there are some potential race conditions in the current management, 
and some inconveniences (e.g. VM name is part of the saved state 
rather than being outside it) which makes e.g. VM fork a bit tricky. 



> > 2) Someone takes the suspended form of Domain A and transfers it to 
> > another machine.  They start it up while Domain A is still running on 
> > another machine (creating a resource conflict).  If you're using DHCP, 
> > VMware can handle this scenario gracefully btw.
> > 
> > Are these errors?  I think it can be argued either way.
> 
> 
> You've raised a good question: are Domain ID's unique only to the system
> they are running on? Do we let the cluster manager or virtual farm
> manager handle Domains per virtual host? How is the process going to be
> managed? Can you have multiple Xen systems in the same network without
> having a cluster manager? What are the policies for migrating from one
> system to another? What if I wish to start a checkpoint of a particular
> domain on another system, which could mean resource conflicts? 
> 
> I'd like to make sure we can easily reference resources to domains to
> hosts, to enable better management - especially on error. 
> 
> These are the questions we need to be answering now, even if we don't
> implement it right away. 

See above: domain IDs are per machine (per Xen instance) things. They 
are low level handles which Xen uses. They are not site wide unique, 
nor are they VM instance unique. 

A sensible management tool strategy would be to allow a set of physical 
nodes to be managed in a coordinated fashion (particularly due to 
live migration, but also due to other forms of location independence, 
e.g. "creating" a VM from a stored (booted) template image). This 
also implies that VM names should be (a) opaque and (b) site-wide unique. 


> > >I disagree. If we're to consider the larger management world, we need to
> > >lay the groundwork for managing domains now. I think the questions
> > >aren't easily answered, but I believe they should be. If we don't
> > >implement everything to start, we should at least have an idea where
> > >we're going. 
> > >  
> > >
> > I completely agree that we need to lay a groundwork.  I think 
> > large-scale domain management tools our outside the scope of the current 
> > focus.  What I'd like to see is an architecture that's good enough that 
> > when these large-scale domain management tools are finally worked out 
> > they can just be plugged in without rewriting the Xen management 
> > infrastructure.
> > 
> > I think the key to this is to keep things as simple as possible.  Don't 
> > require configuration files, don't rely on any sort of internal database 
> > for persistent state.
> 
> 
> I agree that we can't implement the world now, we must do it in small
> steps. But we should start working on an overall architecture with at
> least a stab at some of these questions. We don't want to be surprised a
> year down the road with a situation that our current architecture cannot
> handle. 
> 
> We can start small with incremental releases. The first release (from a
> management standpoint only) could include the xcs layer with xend using
> it. Another incremental release could put the domain configuration file
> into a small store that xend uses through a limited API. This second
> release could include the logic for unique domain ids. The third release
> targets replacing the current Python tools xm/xend with other management
> applications that build upon the initial store. As we add functionality
> and management routines, we add to our store and what it can do.
> Eventually, we enter the cluster management world. 
> 
> Overall, we have a strategy for where we want to go and the uses we want
> it to have. We just split it up into manageable chunks.

Right - and one part of that split is separating out "per Xen instance" 
functionality and "per site" functionality. Most of the xcs and libxc
and other low level bits and pieces are in the former camp; the new 
(in unstable) xend is logically in the latter. 



cheers,

S.


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel