[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 0/2] Re-raise irq_work lost during Xen suspend
- To: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
- From: Julian Vetter <julian@xxxxxxxxxxxxxxxx>
- Date: Mon, 9 Nov 2026 11:16:06 +0100
- Arc-authentication-results: i=1; strato.com; arc=none; dkim=none
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1794219368; s=strato-dkim-0002; d=strato.com; h=In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID:Cc:Date: From:Subject:Sender; bh=z1tFFIkKDjY4xYW1PtNGBh62gyp6wCd07sTLJtfhRp4=; b=iBV54vo/NoN5wM2xVSGV7GLBiw6NcCypt1BkXWz6mix97Uk+SPHYnqvSTND229WaDB zsNAkvxcwnNXir7xfPyu3Z5c11udVYMb4lGVynjkpQeAyLC6p4pCEh9YSatNRBqVTgOg o1JAKJ9vJhq1ld7fN8ufKePFHZP8ttZn/E5zQ70H+FJmIBTYiE6cWlrMXTifZoUFENtV LkWCnP/+emJ8F9hpqAn5lDjPYAT/Iu8SvVD89k9H6LxK/Sj5sIGVghw6X9vDyQW8aWnA HI6HmezT7fdCv9LtEYWXd2zAUYev/r/HOb6boG97XgTSMSuSL6lg9Jfe/uzwYyIboNpy X2Jw==
- Arc-seal: i=1; a=rsa-sha256; t=1794219368; cv=none; d=strato.com; s=strato-dkim-0002; b=Ly5CoXLr6I+bRApne+yZ5VfYEOur9JERXKloz51nArrfXuODkvcUiBvwjOb0RJqqAr y+6zx/tmpuX7jHH2pt1+vR4i94Ff/5k0FgtLYVirrIFegnTjkMsvu7bZ56w5UQHmyeDq 9sRGuKrIVOVSrAhNjANUgeRDFOkCPNKxiV3NBc06yhroqUm7C7gJ+x0zkMrBYCxkvg4R KDzGvZ6NNzkbjmJL6WWXjy53zb9WGUlf73B9oTJ6ZpmtIThVGK0Y04dU9xlrvEDWJUWM 82QNdxK0b/YdFohRzHG8LEidG7z8TmicoOGLi+ZUiZwJwFLGXwgLr1NNSLc9LzJTY1yt PObQ==
- Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=strato-dkim-0002 header.d=outer-limits.org header.i="@outer-limits.org" header.h="In-Reply-To:From:References:Cc:To:Subject:Date:Message-ID"; dkim=permerror header.s=strato-dkim-0003 header.d=outer-limits.org header.i="@outer-limits.org"
- Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, Juergen Gross <jgross@xxxxxxxx>, Thomas Gleixner <tglx@xxxxxxxxxx>, Peter Zijlstra <peterz@xxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>, Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>, Jiayuan Chen <jiayuan.chen@xxxxxxxxx>, "Masami Hiramatsu (Google)" <mhiramat@xxxxxxxxxx>, "Vineeth Pillai (Google)" <vineeth@xxxxxxxxxxxxxxx>, Adrian Hunter <adrian.hunter@xxxxxxxxx>
- Delivery-date: Mon, 09 Nov 2026 10:18:48 +0000
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
On 11/6/26 4:39 PM, Sebastian Andrzej Siewior wrote:
On 2026-11-06 15:57:40 [+0100], Julian Vetter wrote:
We hit this issue with an Ubuntu 26.04 cloudimage guests (7.0 based
kernel with the 8250 as an nbcon console) on XCP-ng 8.3 / Xen 4.17. With
a kernel commandline adding two consoles, i.e., 'console=ttyS1
console=ttyS0'. During a live migration on that setup the printk()s from
gnttab_resume() and xen_irq_resume() queue nbcon_irq_work while the APIC
is disabled. Thus the list transitions from empty to non-empty, but
doesn't receive the IPI to empty the list again. Afterwards the list
stays non-empty, so the IPI never fires and all irq_work piles up.
The Commit 560f4deda327 ("printk/nbcon: Change nbcon_irq_work to
IRQ_WORK_LAZY") removed this particular issue because declaring the IRQ
work to be lazy, causes the timer interrupt to trigger the irq work
instead. But any other non-lazy irq_work queued in that window is still
lost the same way and will cause the list to fill up without being
handled.
Adrian has this:
https://lore.kernel.org/all/20261102074039.26983-2-adrian.hunter@xxxxxxxxx/
I sort of asked for flush on suspend but Adrian pointed out the NMI
problem.
Thanks, I missed that. It is the same issue, just a different trigger.
Adrian's example even ends in the same hung task we see
(fsnotify_mark_destroy_workfn waiting on SRCU). Also, Adrian's
syscore_resume() placement would cover the Xen case as well, as
xen_suspend() calls syscore_resume() after those printk()s.
Looking at #1, you could probably limit it to a nop on
!arch_irq_work_has_interrupt(). The tick should not be disabled if
irq_work_needs_cpu() returns true (you don't add anything).
Yes, right, the lazy part is pointless.
And lazy items get handled via irq_work_tick() anyway. This make me
think if it gets any easier if we handle both in irq_work_tick(). On x86
raised_list is evaluated anyway so by dropping
arch_irq_work_has_interrupt() we would avoid the APIC check and handle
list in this rare case. If the IRQ-work is working then the list remains
empty.
This is maybe a cleaner solution, it needs no APIC access and no call
site in the suspend paths. But irq_work_needs_cpu() would need the same
change, otherwise in a CONFIG_NOHZ=y setting an idle CPU would still sit
on the work until it leaves idle and re-enables the timer tick, no? I
will test this.
Signed-off-by: Julian Vetter <julian.vetter@xxxxxxxxxx>
Sebastian
(Sorry, sending from my personal address, my work mail server currently
mangles mails to the list.)
Julian
|