Re: [Xen-devel] [BUG] Serial port prevents PVH dom0 boot

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Jul 25, 2018 at 04:10:59PM -0400, Connor Davis wrote:
> When booting into dom0=pvh, using the serial console prevents
> dom0 from booting. Serial works until Xen tries to hand off to dom0.
> The specific crash varies with the arguments passed to iommu:
> 
> iommu=1,debug console=com1 com1=115200,8n1
> Xen goes into an infinite loop

You don't need iommu=1, PVH Dom0 already requires an iommu and won't
boot without it.

> iommu=1 console=com1 com1=115200,8n1
> Xen either receives an NMI in process_pending_softirqs or gets

According to the logs below you are manually injecting an NMI by
pressing the 'N' debug key.

> stuck in __sync_local_execstate
> 
> iommu=1
> dom0 boots fine, but USB doesn't work (as others have recently
> posted about). I'm using xen staging and linux xen/tip.
> Example logs for the first two cases are attached.

You also seem to be using UEFI, I have to admit I haven't tried PVH
Dom0 on UEFI yet.

> (XEN) Latest ChangeSet: Wed Jul 25 15:38:23 2018 +0100 git:f776264532
> (XEN) Bootloader: EFI
> (XEN) Command line: dom0_mem=1G,max:1G iommu=1 console=com1 com1=115200,8n1 
> dom0=pvh x2apic=1 nosmp=1

Can you try to boot without the x2apic=1 and nosmp=1 options? Also
please set iommu=debug only.

[...]
> (XEN) WARNING: PVH is an experimental mode with limited functionality
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs

So it gets stuck here while scrubbing the free RAM? It would be good
to dump the CPU state at this point.

> (XEN) ...'A' pressed -> using normal key handling
> (XEN) 'R' pressed -> rebooting machine

[...]
> (XEN) *** Building a PVH Dom0 ***
> (XEN) Triggering NMI on APIC ID 0

This is AFAICT a result of pressing the 'N' debug key. Or at least Xen
thinks the 'N' debug key has been pressed an injects and NMI using
do_nmi_trigger. Can you please confirm whether you have pressed any
debug key?

> (XEN) Uhhuh. NMI received for unknown reason 2c.
> (XEN) Do you have a strange power saving mode enabled?
> (XEN) ----[ Xen-4.12-unstable  x86_64  debug=y   Not tainted ]----
> (XEN) RIP:    e008:[<ffff82d08023b0c2>] _spin_lock+0x1e/0x5c
> (XEN) RFLAGS: 0000000000000006   CONTEXT: hypervisor
> (XEN) rax: 00000000030a030a   rbx: ffff82d080817800   rcx: 0000000000000000
> (XEN) rdx: 0000000000000001   rsi: 0000000000000001   rdi: ffff82d080817806
> (XEN) rbp: ffff82d080827808   rsp: ffff82d0808277f8   r8:  ffff83081bc28000
> (XEN) r9:  0000000000000030   r10: 0000000000000000   r11: 0000000000000010
> (XEN) r12: 0000000000000000   r13: ffff82d080817780   r14: ffff82d080817800
> (XEN) r15: 0000000000000292   cr0: 000000008005003b   cr4: 00000000001526e0
> (XEN) cr3: 00000000be616000   cr2: 0000000000000000
> (XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> 
> (XEN) Xen code around <ffff82d08023b0c2> (_spin_lock+0x1e/0x5c):
> (XEN)  00 00 01 00 f0 0f c1 03 <89> c2 c1 e8 10 66 8b 33 66 39 c6 74 12 48 89 
> d9
> 
> (XEN) Xen stack trace from rsp=ffff82d0808277f8:
> (XEN)    ffff82d080827858 0000000000000296 ffff82d080827828 ffff82d08023b126
> (XEN)    ffff82d080827868 0000000000000000 ffff82d080827858 ffff82d080253789
> (XEN)    000000000000004e ffff82d0808279b8 ffff82d080250a89 ffff82d0808279b8
> (XEN)    ffff82d080827868 ffff82d08025056e ffff82d080827888 ffff82d080216fae
> (XEN)    ffff82d0808279b8 000000000000004e ffff82d080827898 ffff82d08025036a
> (XEN)    ffff82d0808278b8 ffff82d080250aeb ffff82d080817780 ffff82d0808177f8
> (XEN)    ffff82d080827908 ffff82d080252fcb ffff82d080380969 4eff82d08038095d
> (XEN)    ffff82d080380969 ffff82d08093c7a0 ffff82d080817780 ffff82d0808279b8
> (XEN)    0000000000000000 ffff82d0802514d8 ffff82d080827938 ffff82d080251af9
> (XEN)    ffff82d080380969 ffff82d0808279b8 ffff82d080251a9a ffff82d0802514d8
> (XEN)    ffff82d0808279a8 ffff82d0802a87e3 ffff82d080380969 ffff82d0802514da
> (XEN)    ffff82d080380969 ffff82d08038095d ffff82d080380969 0b0f82d08038095d
> (XEN)    ffff82d080380969 ffff8300c03fe000 0000000000000000 0000000000000000
> (XEN)    ffff82d080827fff 0000000000000000 00007d2f7f7d8627 ffff82d080380a2d
> (XEN)    ffff82d08093bc00 00000005c8c55821 ffff82d080817780 ffff82d0802514bb
> (XEN)    ffff82d080827a60 ffff82d08095f300 0000000000000002 0000000601d38f5d
> (XEN)    0000001a9b8e17db ffff82d08095c1c0 ffff82d08095f3a8 0000000000000002
> (XEN)    0000000000000000 ffff82d08093c808 ffff82d080817780 0000000600000000
> (XEN)    ffff82d0802514d8 000000000000e008 0000000000010286 ffff82d080827a60
> (XEN)    0000000000000000 ffff82d080827a90 ffff82d08023df14 0000000000000010
> 
> (XEN) Xen call trace:
> (XEN)    [<ffff82d08023b0c2>] _spin_lock+0x1e/0x5c
> (XEN)    [<ffff82d08023b126>] _spin_lock_irqsave+0x11/0x1b
> (XEN)    [<ffff82d080253789>] serial_end_log_everything+0x37/0x5d
> (XEN)    [<ffff82d08025056e>] console_end_log_everything+0xf/0x18
> (XEN)    [<ffff82d080216fae>] handle_keypress+0x93/0xc9
> (XEN)    [<ffff82d08025036a>] console.c#__serial_rx+0x4f/0x51
> (XEN)    [<ffff82d080250aeb>] console.c#serial_rx+0x62/0x8c
> (XEN)    [<ffff82d080252fcb>] serial_rx_interrupt+0x60/0xb2
> (XEN)    [<ffff82d080251af9>] ns16550.c#__ns16550_poll+0x5f/0xae
> (XEN)    [<ffff82d0802a87e3>] do_invalid_op+0x2b4/0x384
> (XEN)    [<ffff82d080380a2d>] x86_64/entry.S#handle_exception_saved+0x68/0x94
> (XEN)    [<ffff82d0802514d8>] ns16550.c#ns16550_poll+0x1d/0x21
> (XEN)    [<ffff82d08023df14>] timer.c#execute_timer+0x49/0x64
> (XEN)    [<ffff82d08023df9c>] timer.c#timer_softirq_action+0x6d/0x242
> (XEN)    [<ffff82d08023a7a5>] softirq.c#__do_softirq+0x78/0x9a
> (XEN)    [<ffff82d08023a7fa>] process_pending_softirqs+0x33/0x37
> (XEN)    [<ffff82d080613cbd>] pci.c#_setup_hwdom_pci_devices+0x130/0x14b
> (XEN)    [<ffff82d08025b22d>] pci.c#pci_segments_iterate+0x43/0x69
> (XEN)    [<ffff82d080613f93>] setup_hwdom_pci_devices+0x25/0x2c
> (XEN)    [<ffff82d080613fd3>] iommu.c#intel_iommu_hwdom_init+0x39/0x17d
> (XEN)    [<ffff82d0806135bb>] iommu_hwdom_init+0x1b5/0x1d0
> (XEN)    [<ffff82d080631162>] dom0_construct_pvh+0x4a/0x1132
> (XEN)    [<ffff82d0806381e3>] construct_dom0+0x8c/0xcc
> (XEN)    [<ffff82d080628003>] __start_xen+0x21bf/0x26ed
> 
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) FATAL TRAP: vector = 2 (nmi)
> (XEN) [error_code=0000] , IN INTERRUPT CONTEXT
> (XEN) ****************************************

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxxx
> https://lists.xenproject.org/mailman/listinfo/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel