RE: [Xen-devel] [RFC] Add static priority into credit scheduler

["Su, Disheng" <disheng.su@xxxxxxxxx>]

George Dunlap wrote:
> So, just to be clear:  you're proposing that this mechanism *might* be
> useful for a VM with real-time scheduling requirements?  Or are
> actually working on / developing real-time operating systems, and are
> suggesting this in order to support real-time VMs?

The first one, I think it will be useful to consolidate VM with real-time 
requirements:
1. Enterprise real-time: such as SUSE Linux Enterprise Real 
Time(http://www.novell.com/products/realtime/), Red Hat Enterprise 
MRG(http://www.redhat.com/mrg/) and some 
apps/middware(http://www-03.ibm.com/linux/realtime.html). They are used for 
mission-critical applications such as, trading system/VOIP server etc.
2. Embedded real-time: normal usage model is to consolidate one embedded 
RTOS(QNX, VxWorks etc) and a general purpose OS(Linux/Windows) on one cpu core.

> 
> I'm not an expert in real-time scheduling, but it doesn't seem to me
> like this will really be what a real-time system would want.  (Feel
> free to contradict me if you know better.)  It might work OK if there
> were only a single real-time PV guest, but in the face of competition,
> you'd have trouble.  It seems like an actual real-time Xen scheduler
> would want the PV guests to submit deadlines to Xen, and then Xen
> could try to make a decision as to which deadlines to drop if it needs
> to (based on some mechanism).

I agree this is one of way to go. But it's not suitable for all the real time 
OS, e,g. the enterprise real time Linux(no period/deadline at all), whose real 
time scheduling mechanism is SCHED_RR/SCHED_FIFO(based on static priority). 
It's quite different from traditional embedded real time OS. 
On the other hand, there are so many embedded real time COTS OS,  and how about 
unmodified RTOS:)? If you just consolidate one embedded real time OS and one 
general purpose OS(I guess this is the normal usage model currently on 
cellphone/industry control), how about just setting RTOS as highest priority?

It's true, that static priority has problem if two or more RT VM competing with 
each other on one phyiscal cpu. This case can be addressed by real time PV 
guest as you said, or by other ways. Currently I made the assumption that only 
one RT VM and more non-RT VM on one physical cpu core/thread. It's reasonable 
especially with quad/many core.

> 
> The only test you've measured is networking; but networking isn't a
> "real-time" workload, it's a latency-sensitive workload.  And you

Yes, the normal/simple "real-time" workload is "sleep_for_some_ns-and-wake_up", 
such as Cyclictest(http://rt.wiki.kernel.org/index.php/Cyclictest), but it 
depends on hrtimer in dom0. In order to minimize the dependence with dom0, I 
use the assigned network card instead. Sending out a packet from remote 
machine, then test latency according to the response from RT VM. 
It's obvious to see the improvement in scheduler...

> haven't measured:
> * The effect on network traffic if you have several high-priority VMs
> competing 

Currently it's not in my scope. And I think it's very hard to schedule multiple 
RT VM on one CPU in practical.

> * The effect on network traffic of non-prioritized VMs if a
> high-priority VM is receiving traffic, or is misbehaving
> 

RT VM is dealing with critical events, so we trust it...

> You also haven't compared how raising a VM's priority within the
> current credit framework, such as giving it a very high weight,
> affects the numbers.  Can you get similar results if you were to give
> the "latency-sensitive" VMs a weight of, say, 10000, and leave the
> other ones at 256?

"Weight" isn't helpful here.  I had tested one VM with assigned audio device, 
the noise is obvious, if other VM is busy.
The current credit framework has the following issues AFAIK:
        One VM is in OVER state can't be BOOSTed
        Multiple VM in BOOST state, no preemption
So there is no guarantee to schedule which VM in. 

> 
> Overall, I don't think fixed priorities like this is a good solution:
> I think it will create more problems than it solves, and I think it's
> actually harder to predict how a complex system will actually behave
> (and thus harder to configure properly).

static priority is useful in the simple case(one RT VM and multiple non-RT VM 
on one cpu core/thread)
If we trust the RT VM, then it's easy to configure. I mean RT VM is usually 
timely response extern events and then sleep.
I know you may concern about such as RT VM is misbehaving, may monopolise the 
whole cpu.
Static priority is just a scheduling mechanism. It depends on user's favor to 
use it.
Linux supports this kind of static priority also...

> 
> I think the proper solution (and I'm working on a "credit2" scheduler
> that has these properites) is:
> * Fix the credit assignment, so that VMs don't spend very much time
> in "over" 
> * Give VMs that wake up and are under their credits a fixed "boost"
> period (e.g., 1ms)
> * Allow users to specify a cpu "reservation"; so that no matter how
> much work there is on the system, a VM can be guaranteed to get a
> minimum fixed amount of the cpu if it wants it; e.g., dom0 always gets
> 50% of one core if it wants it, no matter how many other VMs are on
> the system.
> 
> #1 and #2 have resulted in significant improvements in TCP throughput
> in the face of competition.  I hope to publish a draft here on the
> list sometime soon, but I'm still working out some of the details.

Glad to know that credit scheduler is being improved. 
If it can improve the latency/real time capabilty with minimal enhancement that 
will be much better:)

> 
Best Regards,
Disheng, Su
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel