Re: [Xen-devel] How Xen handles Dom1 interrupts?

[Daniel Stodden <stodden@xxxxxxxxxx>]

On Fri, 2006-08-11 at 23:13 +0900, Jun Koi wrote:
> Hi Daniel,
> 
> On 8/11/06, Daniel Stodden <stodden@xxxxxxxxxx> wrote:
> > On Fri, 2006-08-11 at 19:03 +0900, Jun Koi wrote:
> > > Good morning,
> > >
> > > I am new to Xen, and trying to understand how Xen works. I have some
> > > questions about virtualization of interrupts on Xen. Thanks in advance
> > > for all helps.
> > >
> > > 1) From papers about Xen architecture found at Xen homepage, Xen
> > > seemingly intercepts interrupts of Dom1 (paravirtualization). So
> > > interrupts occur in Dom1's context are sent to Xen, and Xen handles
> > > them. Is that correct?
> >
> > similar to interrupts to a native operating systems, interrupts could
> > occur in just any context (as long as they're not blocked).
> >
> > the domU never 'receives' the interrupt in a way that it won't even be
> > able to install a handler for it. the idt belongs to xen, and xen is
> > where it is actually handled, interrupting whatever used to be running
> > when it occurred.
> >
> > > But then how can Dom1 gets the interrupts? For example if int3 occurs
> > > inside Dom1, Xen intercepts int3, but then how debugger (like gdb) can
> > > still work in Dom1? Maybe Xen sends that interrupt back to Dom1 after
> > > processing it in VMM layer?
> >
> > hardware interrupts in turn drive 'virtual interrupts', which are in the
> > paravirtualized case a pure software abstraction. in xen, you'll find a
> > lot of references to a concept called 'event channels', and that's just
> > that.
> >
> > so it's xen making then decision on whether a domain interrupt handler
> > is called, which one's called, when it's called, and whether anything
> > called at all. don't worry, in practice, xen is doing the right thing,
> > but that's the way it needs to work. :).
> >
> > > 2) I have read
> > > linux-xen-sparse/arch/i386/kernel/{traps-xen.c,traps.c,vmlinux.lds.S},
> > > and looks like Dom1 still handles interrupts itself? Why, because I
> > > suppose that the VMM already intercepts those interrupts?
> >
> > an guest system, whether it's virtualized or not, is not living in a
> > vacuum. it needs some kind of signalling from the hardware environment,
> > whether it's virtual or physical, in order to get notified of events.
> > that's 'asynchronous input' of some kind, not necessarily purelay
> > hardware interrupt driven, though.
> >
> > e.g. it needs a regular timer interrupt as well as as notification of
> > events regarding virtual device interfaces.
> > so it still has interrupt handlers.
> >
> > it is, however, not modifying the real IDT. see e.g. trap_init() in
> > traps-xen.c and read about HYPERVISOR_set_trap_table in the interface
> > documentations.
> >
> 
> Very interesting. So here is what I imagine about things in Xen:
> 
> - When Dom1 boots, it asks VMM (via set_trap_table hypercall) to setup
> the IDT for it.

should correct myself here, there's a few more calls to know about. may
i point you to the xen interface documentation? the hypercall listings
are comparatively complete regarding that matter.

>  Meanwhile Dom1 still setups its IDT 

in the paravirtualized case, the domains are well aware that there is no
true IDT for them and fully resort to the hypercall interface. which
means they're basically registering a callback. kernel stack setup and
context saving would be similar to a real idt. i must admit that i
didn't look to close at the stack setup. so you'll need to dig through
entry.S yourself :)

> but all the
> interrupts are from event-channels instead of from physical interrupts
> like in native case. An event sent to Dom1 via event-channel will fire
> a virtual interrupt, and Dom1's interrupt handler will handle it like
> in native case.

correct.

> - When an interrupt occurs in Dom1's context, VMM handles it first,
> then fires an event corresponding to the interrupt via event-channel
> to Dom1, thus creates an virtual interrupt. Dom1 processes this
> virtual interrupt.
> 
> Is that all correct?
> 
> 
> I follow the above assumption, and as an example, trying to track down
> how int3 is processed if we use gdb to debug an application in Dom1.
> But looks like the int3 handler in VMM never send event to Dom1 as I
> supposed. Or I missed something? 

do_int3(), which i presume at which you've been looking, is not
signalling the event immediately. what it does is, it sets a flag in the
virtual processor control structure for that guest. this again is
processed upon return from the exception handler. you'll need to dig
trough entry.S again. watch the path follwoing 'jmp error_code' in the
handler. there look for e.g. create_bounce_frame and labels which call
it.

this is due to the fact that a hypervisor can never just 'call back'
into the guest in a synchronous fashion (like a function call). apart
from a whole bunch of reasons why this would not work, it would recurse
with any later interrupt, right? instead, one needs to modify the return
path from the hypervisor to make return to the guest stack look like a
regular exception/interrupt occurring on the real machine. get the idea?

it's indeed a little difficult, if you really want to understand every
bit, you need to know how interrupt and exception handling is
implemented on the processor. means: keep your x86 processor manuals
near.

> Any pointer on which code of VMM does
> this?

for the case of synchronous interrupts (exceptions, int3, etc.) it's all
about entry.S. for the asynchronous case (hardware interrupts), see the
event channel code, which in turn will lead you to the xen scheduler.

regards,
daniel

-- 
Daniel Stodden
LRR     -      Lehrstuhl fÃr Rechnertechnik und Rechnerorganisation
Institut fÃr Informatik der TU MÃnchen             D-85748 Garching
http://www.lrr.in.tum.de/~stodden         mailto:stodden@xxxxxxxxxx
PGP Fingerprint: F5A4 1575 4C56 E26A 0B33  3D80 457E 82AE B0D8 735B

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel