[PATCH 1/2] irq_work: Add irq_work_raise_pending()

[Julian Vetter <julian.vetter@xxxxxxxxxx>]

irq_work_queue() only raises the irq_work interrupt when the per-CPU
list transitions from empty to non-empty. Later queuers rely on that
interrupt to also run their work.

If that interrupt gets lost, nothing runs the raised list on that CPU
anymore. On architectures with an irq_work interrupt irq_work_tick()
skips the raised list, and further irq_work_queue() calls find the list
non-empty and don't raise the interrupt again. All irq_work on that CPU
is stuck.

This happens when irq_work is queued while the local interrupt
controller is disabled, e.g. a self-IPI is dropped when sent while the
x86 local APIC is software-disabled.

Add irq_work_raise_pending(), which re-raises the irq_work interrupt if
work is pending on the current CPU. Code which disables the local
interrupt controller can call it once interrupts are accepted again.

Signed-off-by: Julian Vetter <julian.vetter@xxxxxxxxxx>
---
 include/linux/irq_work.h |  2 ++
 kernel/irq_work.c        | 10 ++++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/linux/irq_work.h b/include/linux/irq_work.h
index 7184e1273965..712a92ead4a5 100644
--- a/include/linux/irq_work.h
+++ b/include/linux/irq_work.h
@@ -62,12 +62,14 @@ void irq_work_run(void);
 void irq_work_run_cpu(unsigned int cpu);
 bool irq_work_needs_cpu(void);
 void irq_work_single(void *arg);
+void irq_work_raise_pending(void);
 
 void arch_irq_work_raise(void);
 
 #else
 static inline bool irq_work_needs_cpu(void) { return false; }
 static inline void irq_work_run(void) { }
+static inline void irq_work_raise_pending(void) { }
 static inline void irq_work_run_cpu(unsigned int cpu) { }
 static inline void irq_work_single(void *arg) { }
 #endif
diff --git a/kernel/irq_work.c b/kernel/irq_work.c
index 73eabcbdcd50..1720a07f12bf 100644
--- a/kernel/irq_work.c
+++ b/kernel/irq_work.c
@@ -112,6 +112,16 @@ static void __irq_work_queue_local(struct irq_work *work)
                irq_work_raise(work);
 }
 
+/* Re-raise the irq_work interrupt on this CPU */
+void irq_work_raise_pending(void)
+{
+       lockdep_assert_irqs_disabled();
+
+       if (!llist_empty(this_cpu_ptr(&raised_list)) ||
+           (tick_nohz_tick_stopped() && 
!llist_empty(this_cpu_ptr(&lazy_list))))
+               arch_irq_work_raise();
+}
+
 /* Enqueue the irq work @work on the current CPU */
 bool irq_work_queue(struct irq_work *work)
 {
-- 
2.53.0



--
Julian Vetter | Vates Hypervisor & Kernel Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech