Re: [Xen-devel] Interrupt injection with ISR set on Intel hardware

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Mon, Oct 29, 2018 at 10:33:11AM -0600, Jan Beulich wrote:
> >>> On 15.10.18 at 12:30, <roger.pau@xxxxxxxxxx> wrote:
> > (XEN)   [22641] PUSH     {sp  0, irq  30, vec 0x21}
> 
> This is the last push or pop.
> 
> > (XEN)   [22650] WAKE     PPR 0x00000020
> > (XEN)                    IRR 
> > 0000000002000000000000000000000000000000000000000000000000000000
> > (XEN)                    ISR 
> > 0000000002000000000000000000000000000000000000000000000000000000
> 
> For one I'm having trouble understanding why IRR here is different
> from ...
> 
> > (XEN) All LAPIC state:
> > (XEN)   [vector]      ISR      TMR      IRR
> > (XEN)   [1f:00]  00000000 00000000 00000000
> > (XEN)   [3f:20]  00000002 00000000 00000000
> > (XEN)   [5f:40]  00000000 00000000 00000000
> > (XEN)   [7f:60]  00000000 00000000 00000000
> > (XEN)   [9f:80]  00000000 00000000 00000000
> > (XEN)   [bf:a0]  00000000 00000000 00000000
> > (XEN)   [df:c0]  00000000 00000000 00000000
> > (XEN)   [ff:e0]  00000000 00000000 04000000
> 
> ... IRR here.

I expect (and this is all hypothesis ATM) that the IRR bit is clear
because the interrupt has been injected, so IRR has been cleared and
ISR has been set (except that ISR was already set).

> > (XEN) Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at 
> > irq.c:1340
> > (XEN) ----[ Xen-4.12-unstable  x86_64  debug=y   Tainted:  C   ]----
> > (XEN) CPU:    1
> > (XEN) RIP:    e008:[<ffff82d08028737d>] do_IRQ+0x8df/0xacb
> > (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
> > (XEN) rax: ffff83086c67202c   rbx: 0000000000000180   rcx: 0000000000000000
> > (XEN) rdx: ffff83086c68ffff   rsi: 000000000000000a   rdi: ffff83086c601e24
> > (XEN) rbp: ffff83086c68fd98   rsp: ffff83086c68fd38   r8:  ffff83086c690000
> > (XEN) r9:  0000000000000030   r10: 0000000004000000   r11: 0000000000000007
> > (XEN) r12: 000000000000011f   r13: 00000000ffffffff   r14: ffff83086c601e00
> > (XEN) r15: ffff82cfffffb100   cr0: 0000000080050033   cr4: 00000000003526e0
> 
> And then I'm having trouble guessing which register holds
> "vector" here: r9 is the only one where I could sort of guess
> it might be a vector, but then the assertion would not have
> triggered. There's in particular no register with the low byte
> being 0x21, nor is there any with it being 0xfa (to match the
> bit that became set in IRR).
> 
> Could you please check or provide the disassembly?

The code chunk that you mention is the following (keep in mind this
has the debug patch attached):

1325    if ( action->ack_type == ACKTYPE_EOI )
1326    {
1327        sp = pending_eoi_sp(peoi);
1328        if ( !((sp == 0) || (peoi[sp-1].vector < vector)) )
1329        {
1330            printk("*** Pending EOI error ***\n");
1331            printk("  cpu #%u, irq %d, vector 0x%x, sp %d\n",
1332                   smp_processor_id(), irq, vector, sp);
1333
1334            dump_peoi_stack(sp);
1335            dump_peoi_records();
1336            dump_lapic();
1337
1338            spin_unlock(&desc->lock);
1339
1340            assert_failed("(sp == 0) || (peoi[sp-1].vector < vector)");

So by the time the code reaches assert_failed the vector value has
long left the registers.

If you want a trace without the debug patch, here it is:

(XEN) Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1173
(XEN) ----[ Xen-4.12-unstable  x86_64  debug=y   Tainted:  C   ]----
(XEN) CPU:    7
(XEN) RIP:    e008:[<ffff82d08028737c>] do_IRQ+0x496/0x680
(XEN) RFLAGS: 0000000000010046   CONTEXT: hypervisor
(XEN) rax: ffff83085609e4c0   rbx: ffff83086c601e00   rcx: 0000000000000001
(XEN) rdx: 0000000000000021   rsi: 0000000000000021   rdi: 0000000000000001
(XEN) rbp: ffff830856097d98   rsp: ffff830856097d38   r8:  0000000000000021
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: ffff83086c6f1f10   r13: 0000000000000021   r14: ffff83086c601e00
(XEN) r15: 000000000000001e   cr0: 0000000080050033   cr4: 00000000003526e0
(XEN) cr3: 0000000855ba7000   cr2: 000055c6e4a1c0f0
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008

In this case there are several registers that contain 0x21.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel