Re: [Xen-users] Xen resource guarantees

[Tim Post <tim.post@xxxxxxxxxxxxxxx>]

On Fri, 2006-12-01 at 17:16 +0100, Henning Sprang wrote:

> For the Disk I/O I also came to the conclusion, that, quite different
> from Network I/O, where it's more or less some math, and don't forget
> it's influenced by CPU, it's hard to calculate the maximum possible, and
> so it's hard to really know about saturation for sure.

I think like any other kind of scheduler one would have to be able to
assign some kind of weight to each, but in this case it would be
reversed from what you would think.

Faster drives would be a lower weight, where slower drives would be a
heavier weight. Ideally any SAN or NAS could contain both.

This is surely one of those cases where talking about it is *much*
easier than accomplishing it :) 

> 
> The only possible thing would be to really measure maximum outpout for
> each (hardware/software) configuration.
> 

Yes, measured:expected ratios could help auto-determine the weights I
mentioned above based on the drive type. What's going to be hard to
detect are caching controllers, PCI / SODIMM cache cards (on board or
otherwise), etc .. however discover is usually pretty good at picking
those up if the latest data is installed. 

Count on someone resurrecting a RLL controller and drive from their
closet just to see what happens.

Possible, yes .. easy , hardly.

> This is really for knowing about saturation - for load balancing a
> similar way as described in the XenMon papers might be possible.
> This sound very interesting, but this is currently out of reach for my
> time-ressources.

I don't think *anyone* has time to tackle this unless they were paid to
work on it full time, or could donate many many hours to an open source
initiative. You'd also need some rather expensive NAS/SAN sandboxes.

I can tell you how it *should* work .. and hope I didn't annoy you to
the point that you don't want to make it anymore. I'm an admin, not a
programmer - I do much better adapting things other people make and
suggesting improvements. Bash -> good. C -> not so good - I get by.

I do think we'll reach a point where it becomes a must have .. and most
(GNU/GPL) things worth while are created out of necessity. My hope is
IBM or AMD picks this up and runs with it, they have much more money to
throw at it than we do :) 10-15K would get it done if spent wisely
(remember, you have about a dozen different RAID drivers to patch, to
start) .. then you have the generic IDE and SATA if you go that route.

It also dawned on me that inode allocation in ext3, jfs, ocfs2 as well
as gfs could be scheduled and delayed a few miliseconds .. that would
also accomplish this provided you sent dirty pages to somewhere other
than the volume being written... But now you're going and modifying file
systems. 

I think this would be easier in cluster file systems such as gfs or
ocfs2, especially ocfs2 .. the voting mechanism may lend well to it.

I have yet to play with CLVM enough to mention it. I have not kept up on
XenFS and its capabilities.. this may be in the works?

Now make the above tolerant if some nitwit hits the re-set button, or a
UPS dies under load, etc.. ext3 (and other journaling file systems, I'm
not picking on ext3) under the best of circumstances is unpredictable
when that happens - (again, depending on the media type and controller).

No matter how you slice it, this is major surgery.

There's just too many ideas that can be thrown into it .. and with them
come too many egos. Only a funded corporate project management effort
could bring a release of anything useful to fruition any time soon, or
perhaps a pack of bored Buddhist programmers. Anyone is, of course
welcome to prove me wrong with a release :D

> 
> Henning
> 

Best as always,
-Tim


------------
Disclaimer : The above post contains run on sentences. The author is not
responsible should you find yourself cross eyed, or riddled with sudden
unexpected violent urges. Should this happen, please refrain from all
interaction with pets, spouses and children until symptoms subside.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users