Re: [Xen-devel] Notes on stubdoms and latency on ARM

[Volodymyr Babchuk <vlad.babchuk@xxxxxxxxx>]

Hi Stefano,

On 18 May 2017 at 22:00, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:

> Description of the problem: need for a place to run emulators and
> mediators outside of Xen, with low latency.
>
> Explanation of what EL0 apps are. What should be their interface with
> Xen? Could the interface be the regular hypercall interface? In that
> case, what's the benefit compared to stubdoms?
I imagined this as separate syscall interface (with finer policy
rules). But this can be discussed, of course.

> The problem with stubdoms is latency and scheduling. It is not
> deterministic. We could easily improve the null scheduler to introduce
> some sort of non-preemptive scheduling of stubdoms on the same pcpus of
> the guest vcpus. It would still require manually pinning vcpus to pcpus.
I see couple of other problems with stubdoms. For example, we need
mechanism to load mediator stubdom before dom0.

> Then, we could add a sched_op hypercall to let the schedulers know that
> a stubdom is tied to a specific guest domain.
What if one stubdom serves multiple domains? This is TEE use case.

> The other issue with stubdoms is context switch times. Volodymyr showed
> that minios has much higher context switch times compared to EL0 apps.
> It is probably due to GIC context switch, that is skipped for EL0 apps.
> Maybe we could skip GIC context switch for stubdoms too, if we knew that
> they are not going to use the VGIC. At that point, context switch times
> should be very similar to EL0 apps.
So you are suggesting to create something like lightweight stubdom. I
generally like this idea. But AFAIK, vGIC is used to deliver events
from hypervisor to stubdom. Do you want to propose another mechanism?
Also, this is sounds much like my EL0 PoC :)

> ACTIONS:
> Improve the null scheduler to enable decent stubdoms scheduling on
> latency sensitive systems.
I'm not very familiar with XEN schedulers. Looks like null scheduler
is good for hard RT, but isn't fine for a generic consumer system. How
do you think: is it possible to make credit2 scheduler to schedule
stubdoms in the same way?

> Investigate ways to improve context switch times on ARM.
Do you have any tools to profile or trace XEN core? Also, I don't
think that pure context switch time is the biggest issue. Even now, it
allows 180 000 switches per second (if I'm not wrong). I think,
scheduling latency is more important.

-- 
WBR Volodymyr Babchuk aka lorc [+380976646013]
mailto: vlad.babchuk@xxxxxxxxx

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel