Xen project Mailing List

Re: Lockdep show 6.6-rc regression in Xen HVM CPU hotplug

To: Juergen Gross <jgross@xxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, paulmck <paulmck@xxxxxxxxxx>

From: David Woodhouse <dwmw2@xxxxxxxxxxxxx>

Date: Tue, 24 Oct 2023 14:45:03 +0100

Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>, Rahul Singh <rahul.singh@xxxxxxx>, Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>, Thomas Gleixner <tglx@xxxxxxxxxxxxx>

Delivery-date: Tue, 24 Oct 2023 13:45:14 +0000

List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Tue, 2023-10-24 at 14:08 +0200, Juergen Gross wrote: > > > I can probably change xen_send_IPI_one() to not need irq_get_chip_data(). > > David, could you test the attached patch, please? Build tested only. No longer whines when offlining CPU1. Still triple-faults when bringing it back online. Or if I remove the lockdep_assert_irqs_disabled() from load_current_idt(), gives the same warnings as before, all on the same theme about IRQs being enabled when they shouldn't be. [root@localhost cpu1]# echo 1 > online [ 26.435377] installing Xen timer for CPU 1 [ 26.437493] smpboot: Booting Node 0 Processor 1 APIC 0x2 [ 26.438830] ------------[ cut here ]------------ [ 26.438842] WARNING: CPU: 1 PID: 0 at arch/x86/kernel/cpu/common.c:454 cr4_update_irqsoff+0x2e/0x60 [ 26.438858] Modules linked in: [ 26.438864] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.6.0-rc5+ #1382 [ 26.438870] Hardware name: Xen HVM domU, BIOS 4.17.1 09/26/2023 [ 26.438873] RIP: 0010:cr4_update_irqsoff+0x2e/0x60 [ 26.438881] Code: 89 f7 65 48 8b 15 ba 07 fd 7d 8b 0d 54 28 dc 01 85 c9 74 18 65 8b 0d 51 f5 fb 7d 85 c9 75 0d 65 8b 0d 42 f3 fb 7d 85 c9 74 02 <0f> 0b 48 f7 d7 48 21 d7 48 09 c7 48 39 fa 75 05 c3 cc cc cc cc 65 [ 26.438885] RSP: 0000:ffffb245000aff10 EFLAGS: 00010002 [ 26.438891] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000001 [ 26.438895] RDX: 00000000001300a0 RSI: 000000000000000f RDI: 000000000000000f [ 26.438898] RBP: ffff9002c1398000 R08: 00000000fffffe51 R09: 0000000000000000 [ 26.438902] R10: ffff90030e106078 R11: 0000000000000000 R12: 0000000000000001 [ 26.438905] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 26.438908] FS: 0000000000000000(0000) GS:ffff90030e100000(0000) knlGS:0000000000000000 [ 26.438912] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 26.438916] CR2: 0000000000000000 CR3: 000000004483c000 CR4: 00000000001300a0 [ 26.438923] Call Trace: [ 26.438927] <TASK> [ 26.438931] ? cr4_update_irqsoff+0x2e/0x60 [ 26.438937] ? __warn+0x81/0x170 [ 26.438951] ? cr4_update_irqsoff+0x2e/0x60 [ 26.438959] ? report_bug+0x18d/0x1c0 [ 26.438974] ? handle_bug+0x3c/0x80 [ 26.438983] ? exc_invalid_op+0x13/0x60 [ 26.438990] ? asm_exc_invalid_op+0x16/0x20 [ 26.439015] ? cr4_update_irqsoff+0x2e/0x60 [ 26.439024] cpu_init+0x54/0x1a0 [ 26.439034] start_secondary+0x2e/0x140 [ 26.439044] secondary_startup_64_no_verify+0x178/0x17b [ 26.439071] </TASK> [ 26.439074] irq event stamp: 109567 [ 26.439077] hardirqs last enabled at (109567): [<ffffffff82720a76>] acpi_idle_play_dead+0x46/0x70 [ 26.439087] hardirqs last disabled at (109566): [<ffffffff82124e00>] do_idle+0x90/0xe0 [ 26.439094] softirqs last enabled at (109502): [<ffffffff82cec89c>] __do_softirq+0x31c/0x423 [ 26.439105] softirqs last disabled at (109489): [<ffffffff820bab91>] __irq_exit_rcu+0x91/0x100 [ 26.439111] ---[ end trace 0000000000000000 ]--- [ 26.439141] ------------[ cut here ]------------ [ 26.439145] WARNING: CPU: 1 PID: 0 at kernel/rcu/tree.c:4429 rcu_cpu_starting+0x192/0x1f0 [ 26.439159] Modules linked in: [ 26.439164] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 6.6.0-rc5+ #1382 [ 26.439168] Hardware name: Xen HVM domU, BIOS 4.17.1 09/26/2023 [ 26.439171] RIP: 0010:rcu_cpu_starting+0x192/0x1f0 [ 26.439179] Code: ff 65 8b 05 80 7c e9 7d 85 c0 75 af 65 8b 05 71 7a e9 7d 85 c0 74 a4 0f 0b eb a0 65 8b 05 62 7a e9 7d 85 c0 0f 84 94 fe ff ff <0f> 0b e9 8d fe ff ff 89 c6 4c 89 f7 e8 9d 71 b6 00 90 e9 c0 fe ff [ 26.439182] RSP: 0000:ffffb245000aff08 EFLAGS: 00010002 [ 26.439188] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000000 [ 26.439191] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001 [ 26.439194] RBP: 0000000000000000 R08: 000000000383c000 R09: 0000000000000000 [ 26.439197] R10: ffff90030e106078 R11: 0000000000000000 R12: 0000000000000000 [ 26.439200] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 26.439203] FS: 0000000000000000(0000) GS:ffff90030e100000(0000) knlGS:0000000000000000 [ 26.439206] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 26.439209] CR2: 0000000000000000 CR3: 000000004483c001 CR4: 00000000001706a0 [ 26.439213] Call Trace: [ 26.439215] <TASK> [ 26.439218] ? rcu_cpu_starting+0x192/0x1f0 [ 26.439225] ? __warn+0x81/0x170 [ 26.439234] ? rcu_cpu_starting+0x192/0x1f0 [ 26.439244] ? report_bug+0x18d/0x1c0 [ 26.439256] ? handle_bug+0x3c/0x80 [ 26.439262] ? exc_invalid_op+0x13/0x60 [ 26.439269] ? asm_exc_invalid_op+0x16/0x20 [ 26.439290] ? rcu_cpu_starting+0x192/0x1f0 [ 26.439306] start_secondary+0x3f/0x140 [ 26.439314] secondary_startup_64_no_verify+0x178/0x17b [ 26.439338] </TASK> [ 26.439340] irq event stamp: 109567 [ 26.439343] hardirqs last enabled at (109567): [<ffffffff82720a76>] acpi_idle_play_dead+0x46/0x70 [ 26.439349] hardirqs last disabled at (109566): [<ffffffff82124e00>] do_idle+0x90/0xe0 [ 26.439354] softirqs last enabled at (109502): [<ffffffff82cec89c>] __do_softirq+0x31c/0x423 [ 26.439362] softirqs last disabled at (109489): [<ffffffff820bab91>] __irq_exit_rcu+0x91/0x100 [ 26.439367] ---[ end trace 0000000000000000 ]--- [ 26.439377] ------------[ cut here ]------------ [ 26.439418] WARNING: CPU: 1 PID: 0 at kernel/rcu/tree.c:4462 rcu_cpu_starting+0x17f/0x1f0 [ 26.439427] Modules linked in: [ 26.439432] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 6.6.0-rc5+ #1382 [ 26.439436] Hardware name: Xen HVM domU, BIOS 4.17.1 09/26/2023 [ 26.439438] RIP: 0010:rcu_cpu_starting+0x17f/0x1f0 [ 26.439446] Code: c5 60 dc 55 83 b8 04 00 00 00 f0 0f c1 45 00 e9 23 ff ff ff 65 8b 05 80 7c e9 7d 85 c0 75 af 65 8b 05 71 7a e9 7d 85 c0 74 a4 <0f> 0b eb a0 65 8b 05 62 7a e9 7d 85 c0 0f 84 94 fe ff ff 0f 0b e9 [ 26.439449] RSP: 0000:ffffb245000aff08 EFLAGS: 00010002 [ 26.439454] RAX: 0000000000000001 RBX: ffff90030e12f280 RCX: 400000000000177a [ 26.439458] RDX: 3fffffffffffffff RSI: ffffffff8344ef66 RDI: ffffffff834cb177 [ 26.439461] RBP: 0000000000029328 R08: 000000000000003e R09: 000000000000003e [ 26.439465] R10: 0000000000000000 R11: ffff9002c1398d48 R12: ffffffff83c03d00 [ 26.439468] R13: 0000000000000002 R14: ffffffff83cb7280 R15: 0000000000000000 [ 26.439471] FS: 0000000000000000(0000) GS:ffff90030e100000(0000) knlGS:0000000000000000 [ 26.439475] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 26.439478] CR2: 0000000000000000 CR3: 000000004483c001 CR4: 00000000001706a0 [ 26.439481] Call Trace: [ 26.439483] <TASK> [ 26.439485] ? rcu_cpu_starting+0x17f/0x1f0 [ 26.439492] ? __warn+0x81/0x170 [ 26.439502] ? rcu_cpu_starting+0x17f/0x1f0 [ 26.439512] ? report_bug+0x18d/0x1c0 [ 26.439525] ? handle_bug+0x3c/0x80 [ 26.439530] ? exc_invalid_op+0x13/0x60 [ 26.439537] ? asm_exc_invalid_op+0x16/0x20 [ 26.439559] ? rcu_cpu_starting+0x17f/0x1f0 [ 26.439570] ? rcu_cpu_starting+0xef/0x1f0 [ 26.439582] start_secondary+0x3f/0x140 [ 26.439590] secondary_startup_64_no_verify+0x178/0x17b [ 26.439616] </TASK> [ 26.439618] irq event stamp: 109567 [ 26.439620] hardirqs last enabled at (109567): [<ffffffff82720a76>] acpi_idle_play_dead+0x46/0x70 [ 26.439626] hardirqs last disabled at (109566): [<ffffffff82124e00>] do_idle+0x90/0xe0 [ 26.439631] softirqs last enabled at (109502): [<ffffffff82cec89c>] __do_softirq+0x31c/0x423 [ 26.439639] softirqs last disabled at (109489): [<ffffffff820bab91>] __irq_exit_rcu+0x91/0x100 [ 26.439644] ---[ end trace 0000000000000000 ]--- [ 26.443877] cpu 1 spinlock event irq 57

Attachment: smime.p7s
Description: S/MIME cryptographic signature

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.