Re: IRQ latency measurements in hypervisor

[Dario Faggioli <dfaggioli@xxxxxxxx>]

On Thu, 2021-01-21 at 01:20 +0000, Volodymyr Babchuk wrote:
> Hi Dario,
> 
Hi :-)

> Dario Faggioli writes:
> > Anyway, as I was saying, having a latency which is ~ 2x of your
> > period
> > is ok, and it should be expected (when you size the period). In
> > fact,
> > let's say that your budget is Xus, and your period is 30us. This
> > means
> > that you get to execute for Xus every 30us. So, basically, at time
> > t0
> > you are given a budget of Xus and you are guaranteed to be able to
> > use
> > it all within time t1=t0+30us. At that time (t1=t0+30us) you are
> > given
> > another Xus amount of budget, and you are guaranteed to be able to
> > use
> > it all within t2=t1+30us=t0+60us.
> 
> Well, I'm not sure if I got you right. Are you saying that unused
> budget
> is preserved?
> 
No, it's not preserved and that's not what I meant... sorry if I did
not manage to make myself clear.

> If I understood it correct, any unused budget is lost. So, basically
> RDTS guarantees that your vcpu will able to run Xus in total every
> Yus,
> where X is the budget and Y is the period. 
>
Yep. Every Y, you are given X and you can use it. When, after Y, a new
replenishment happens, you are given X again, so if you did not use all
of X during the previous period, the amount you didn't use is lost.

> Also, it does not guarantee
> that vCPU will be scheduled right away, so for period of 100us and
> budget of 30us it will be perfectly fine to have latency of 70+us:
> 
Exactly, that kind of more what I was trying to say myself.

In fact, as you say, with period 100 and budget 30, it is perfectly
"legal" for the vcpu to start executing at 70 after the replenishment
(i.e., after the beginning of a new period).

And that is also why, theoretically, the maximum latency between to
consecutive execution of a vcpu could be, in the worst possible case,
2*P-2*B (where, this time, P is the period and B is the budget).

> If at t=0 new period begins and in the same time IRQ fires, but RDTS
> decides to run another task, it is possible that your vCPU will be
> scheduled at only t=70us.
>
Exactly.

> > What it will therefore record would be a latency to t2-5us, which
> > in
> > fact is:
> > 
> >   t1 + 30us - 5us = t0 + 30us + 30us - 5us =
> > = 0 + 60us - 5us = 55us ~= 60us
> > 
> > So... May this be the case?
> 
> Yes, probably this is the issue. I used budget of 15us in this
> case. But, taking into account that minimal observed latency is 3us
> and
> typical is ~10us, it is quite possible that budget will be emptied
> before IRQ handler will have a chance to complete.
> 
Indeed.

> It would be great to have priorities in RTDS, so more critical task
> can
> preempt less critical one. 
>
Well, I see from where this comes but it's a dynamic scheduler, so the
priority is given by the (absolute) deadline. The challenge, therefore,
is to determine the proper period and budget to be assigned to each
vcpu, given your workload and your requirement.

With fixed priority schedulers, this "modelling" part could be easier,
but then the scheduler themselves have most of the time worse
performance and only allow you to utilize a fraction of the resources
that you have available.

And adding some kind of priority on top of a dynamic scheduler such as
RTDS --although it may sound appealing when thinking about it-- would
mess up both the theoretical analysis and the implementation of the
algorithm itself, believe me. :-)

> I believe, I have seen corresponding patches
> somewhere...
> 
Mmm... not sure. Maybe you're referring to how "extratime" is handled?
Because there is a concept of priority in the way it's handled, but
it's mostly an implementation detail.

That being said, was extratime enabled, in your experiments, for the
various vcpus?

Of course, it's not that you can rely on a particular vcpu to be able
to take advantage of it, because it's not how it works. But at the same
time it does not disrupt the real-time guarantees so it's safe to
use... It may be interesting to give it a try.

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part