[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v1 0/2] xen/arm: maintenance_interrupt SMP fix



> Right, that's why changing it to cpumask_of(0) shouldn't make any
> difference for xen-unstable (it should make things clearer, if nothing
> else) but it should fix things for Oleksandr.

Unfortunately, it is not enough for stable work.

I was tried to use cpumask_of(smp_processor_id()) instead of cpumask_of(0) in
gic_route_irq_to_guest(). And as result, I don't see our situation
which cause to deadlock in on_selected_cpus function (expected).
But, hypervisor sometimes hangs somewhere else (I have not identified
yet where this is happening) or I sometimes see traps, like that:
("WARN_ON(p->desc != NULL)" in maintenance_interrupt() leads to them)

(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4-unstable  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     00242c1c __warn+0x20/0x28
(XEN) CPSR:   200001da MODE:Hypervisor
(XEN)      R0: 0026770c R1: 00000001 R2: 3fd2fd00 R3: 00000fff
(XEN)      R4: 00406100 R5: 40020ee0 R6: 00000000 R7: 4bfdf000
(XEN)      R8: 00000001 R9: 4bfd7ed0 R10:00000001 R11:4bfd7ebc R12:00000002
(XEN) HYP: SP: 4bfd7eb4 LR: 00242c1c
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 00020000dec6a000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 00000000000028b5
(XEN)  TTBR0_EL2: 00000000d2014000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 0000000000482110
(XEN)      HDFAR: fa211190
(XEN)      HIFAR: 00000000
(XEN)
(XEN) Xen stack trace from sp=4bfd7eb4:
(XEN)    0026431c 4bfd7efc 00247a54 00000024 002e6608 002e6608 00000097 00000001
(XEN)    00000000 4bfd7f54 40017000 40005f60 40017014 4bfd7f58 00000019 00000000
(XEN)    40005f60 4bfd7f24 00248e60 00000009 00000019 00404000 4bfd7f58 00000000
(XEN)    00405000 000045f0 002e7694 4bfd7f4c 00248978 c0079a90 00000097 00000097
(XEN)    00000000 fa212000 ea80c900 00000001 c05b8a60 4bfd7f54 0024f4b8 4bfd7f58
(XEN)    00251830 ea80c950 00000000 00000001 c0079a90 00000097 00000097 00000000
(XEN)    fa212000 ea80c900 00000001 c05b8a60 00000000 e9879e3c ffffffff b6efbca3
(XEN)    c03b29fc 60000193 9fffffe7 b6c0bbf0 c0607500 c03b3140 e9879eb8 c007680c
(XEN)    c060750c c03b32c0 c0607518 c03b3360 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 3ff6bebf a0000113 800b0193 800b0093 40000193 00000000
(XEN)    ffeffbfe fedeefff fffd5ffe
(XEN) Xen call trace:
(XEN)    [<00242c1c>] __warn+0x20/0x28 (PC)
(XEN)    [<00242c1c>] __warn+0x20/0x28 (LR)
(XEN)    [<00247a54>] maintenance_interrupt+0xfc/0x2f4
(XEN)    [<00248e60>] do_IRQ+0x138/0x198
(XEN)    [<00248978>] gic_interrupt+0x58/0xc0
(XEN)    [<0024f4b8>] do_trap_irq+0x10/0x14
(XEN)    [<00251830>] return_from_trap+0/0x4
(XEN)

Also I am posting maintenance_interrupt() from my tree:

static void maintenance_interrupt(int irq, void *dev_id, struct
cpu_user_regs *regs)
{
    int i = 0, virq, pirq;
    uint32_t lr;
    struct vcpu *v = current;
    uint64_t eisr = GICH[GICH_EISR0] | (((uint64_t) GICH[GICH_EISR1]) << 32);

    while ((i = find_next_bit((const long unsigned int *) &eisr,
                              64, i)) < 64) {
        struct pending_irq *p, *n;
        int cpu, eoi;

        cpu = -1;
        eoi = 0;

        spin_lock_irq(&gic.lock);
        lr = GICH[GICH_LR + i];
        virq = lr & GICH_LR_VIRTUAL_MASK;

        p = irq_to_pending(v, virq);
        if ( p->desc != NULL ) {
            p->desc->status &= ~IRQ_INPROGRESS;
            /* Assume only one pcpu needs to EOI the irq */
            cpu = p->desc->arch.eoi_cpu;
            eoi = 1;
            pirq = p->desc->irq;
        }
        if ( !atomic_dec_and_test(&p->inflight_cnt) )
        {
            /* Physical IRQ can't be reinject */
            WARN_ON(p->desc != NULL);
            gic_set_lr(i, p->irq, GICH_LR_PENDING, p->priority);
            spin_unlock_irq(&gic.lock);
            i++;
            continue;
        }

        GICH[GICH_LR + i] = 0;
        clear_bit(i, &this_cpu(lr_mask));

        if ( !list_empty(&v->arch.vgic.lr_pending) ) {
            n = list_entry(v->arch.vgic.lr_pending.next, typeof(*n), lr_queue);
            gic_set_lr(i, n->irq, GICH_LR_PENDING, n->priority);
            list_del_init(&n->lr_queue);
            set_bit(i, &this_cpu(lr_mask));
        } else {
            gic_inject_irq_stop();
        }
        spin_unlock_irq(&gic.lock);

        spin_lock_irq(&v->arch.vgic.lock);
        list_del_init(&p->inflight);
        spin_unlock_irq(&v->arch.vgic.lock);

        if ( eoi ) {
            /* this is not racy because we can't receive another irq of the
             * same type until we EOI it.  */
            if ( cpu == smp_processor_id() )
                gic_irq_eoi((void*)(uintptr_t)pirq);
            else
                on_selected_cpus(cpumask_of(cpu),
                                 gic_irq_eoi, (void*)(uintptr_t)pirq, 0);
        }

        i++;
    }
}


Oleksandr Tyshchenko | Embedded Developer
GlobalLogic

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.