RE: [Xen-devel] xend leaks/bugs/etc

[Allen Short <washort@xxxxxxxxxx>]

On Sun, 2005-04-17 at 16:42 +0100, Ian Pratt wrote: 

> Allen, I think we've come to the conclusion that Twisted was rather
> overkill for our needs, and led to some rather confusing code that has
> proved hard to maintain.

With all due respect, I work on 5 projects that use Twisted and,
overall, they're the easiest codebases to extend that I've dealt with.
xend's code is by far some of the worst Python code I've worked on.

>  I've no doubt that someone more experienced
> with using Twisted could have done a better job, but do you really think
> it's the best route forward?  Xend is a 'control plane' daemon and
> doesn't have to handle a high rate of invocations.

This is a point in favor of Twisted, I'd think; if you needed very high
performance in that area, Python might not be appropriate.

> It needs some ability to handle asynchronous or out-of-order events, but 
> this could be handled by simple language-level threads (we don't need 
> concurrency). 

Given the current architecture (a daemon that accepts connections from a
commandline tool or from a web interface), it would seem that you do
need concurrency; personally, I'd find it inconvenient if this was
handled differently. Plus, the languages that I'm familiar with that
provide language-level threads require at least as much
infrastructure/resource usage as Python.

> 
> The other downside of using Twisted is that its not available in some
> distros, and we've had a few issues with version mismatches.

Other projects (such as Chandler) have dealt with this by shipping
Twisted in their release tarballs. I believe this strategy would be
reasonable for Xen, especially now that Twisted has split some of its
less-used subprojects into separate packages. 


>  It also has quite an impact on the RSS memory footprint, which is not ideal.

I believe that the memory footprint can be significantly reduced; the
current codebase seems to have a good deal of unnecessary complexity.

> What do you think?

I think that there are probably some Xen deployments that would benefit
from a minimal-functionality, minimal-resource-usage control daemon, but
that they are not the only use case. The project that led to my interest
in Xen is a good example: I want to do dynamic auction-based resource
allocation to domains, a la Miller and Drexler's "Incentive Engineering
for Computational Resource Management". 
(http://www.agorics.com/Library/agoricpapers.html ) This would be most
easily achieved by putting my auction/resource-allocation code in the
same process as xend. Unfortunately its current implementation makes
that prohibitively difficult -- my current prototype uses the HTTP
interface, with some difficulty. 

Given these concerns -- greater flexibility, lower memory usage, more
comprehensible code -- I believe my best choice is to reimplement xend,
using the existing lowlevel xc and xu modules. I need it for the things
I want to write anyway, but hopefully enough configuration/UI
compatibility can be maintained for it to be useful to the community.

Allen

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel