Dear Xen experts,
Would greatly appreciate some hints on how to move forward with this one…
I have included further details on Xen diagnostics below:
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 00000000e1000000
(XEN) Allocating 1:1 mappings totalling 512MB for dom0:
(XEN) BANK[0] 0x000000a0000000-0x000000c0000000 (512MB)
(XEN) Grant table range: 0x000000fec00000-0x000000fec60000
(XEN) Loading zImage from 00000000e1000000 to 00000000a0080000-00000000a223c808
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading dom0 DTB to 0x00000000a8000000-0x00000000a803435c
(XEN) Scrubbing Free RAM on 1 nodes using 4 CPUs
(XEN) ........done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: All
(XEN) ***************************************************
(XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) This option is intended to aid debugging of Xen by ensuring
(XEN) that all output is synchronously delivered on the serial line.
(XEN) However it can introduce SIGNIFICANT latencies and affect
(XEN) timekeeping. It is NOT recommended for production use!
(XEN) ***************************************************
(XEN) 3... 2... 1...
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 300kB init memory.
(XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch input to DOM0)
(XEN) 'h' pressed -> showing installed handlers
(XEN) key '%' (ascii '25') => trap to xendbg
(XEN) key '*' (ascii '2a') => print all diagnostics
(XEN) key '0' (ascii '30') => dump Dom0 registers
(XEN) key 'A' (ascii '41') => toggle alternative key handling
(XEN) key 'H' (ascii '48') => dump heap info
(XEN) key 'R' (ascii '52') => reboot machine
(XEN) key 'a' (ascii '61') => dump timer queues
(XEN) key 'd' (ascii '64') => dump registers
(XEN) key 'e' (ascii '65') => dump evtchn info
(XEN) key 'g' (ascii '67') => print grant table usage
(XEN) key 'h' (ascii '68') => show this message
(XEN) key 'm' (ascii '6d') => memory info
(XEN) key 'q' (ascii '71') => dump domain (and guest debug) info
(XEN) key 'r' (ascii '72') => dump run queues
(XEN) key 't' (ascii '74') => display multi-cpu clock info
(XEN) key 'w' (ascii '77') => synchronously dump console ring buffer (dmesg)
(XEN) '*' pressed -> firing all diagnostic keyhandlers
(XEN) [d: dump registers]
(XEN) 'd' pressed -> dumping registers
(XEN)
(XEN) *** Dumping CPU0 guest state (d0v0): ***
(XEN) ----[ Xen-4.8.5 arm64 debug=n Tainted: C ]----
(XEN) CPU: 0
(XEN) PC: 00000000a0080000
(XEN) LR: 0000000000000000
(XEN) SP_EL0: 0000000000000000
(XEN) SP_EL1: 0000000000000000
(XEN) CPSR: 000001c5 MODE:64-bit EL1h (Guest Kernel, handler)
(XEN) X0: 00000000a8000000 X1: 0000000000000000 X2: 0000000000000000
(XEN) X3: 0000000000000000 X4: 0000000000000000 X5: 0000000000000000
(XEN) X6: 0000000000000000 X7: 0000000000000000 X8: 0000000000000000
(XEN) X9: 0000000000000000 X10: 0000000000000000 X11: 0000000000000000
(XEN) X12: 0000000000000000 X13: 0000000000000000 X14: 0000000000000000
(XEN) X15: 0000000000000000 X16: 0000000000000000 X17: 0000000000000000
(XEN) X18: 0000000000000000 X19: 0000000000000000 X20: 0000000000000000
(XEN) X21: 0000000000000000 X22: 0000000000000000 X23: 0000000000000000
(XEN) X24: 0000000000000000 X25: 0000000000000000 X26: 0000000000000000
(XEN) X27: 0000000000000000 X28: 0000000000000000 FP: 0000000000000000
(XEN)
(XEN) ELR_EL1: 0000000000000000
(XEN) ESR_EL1: 00000000
(XEN) FAR_EL1: 0000000000000000
(XEN)
(XEN) SCTLR_EL1: 00c50838
(XEN) TCR_EL1: 00000000
(XEN) TTBR0_EL1: 0000000000000000
(XEN) TTBR1_EL1: 0000000000000000
(XEN)
(XEN) VTCR_EL2: 80043594
(XEN) VTTBR_EL2: 000100017f0f9000
(XEN)
(XEN) SCTLR_EL2: 30cd183d
(XEN) HCR_EL2: 000000008038663f
(XEN) TTBR0_EL2: 00000000fecfc000
(XEN)
(XEN) ESR_EL2: 8200000d
(XEN) HPFAR_EL2: 0000000000000000
(XEN) FAR_EL2: 00000000a0080000
(XEN)
(XEN) Guest stack trace from sp=0:
(XEN) Failed to convert stack to physical address
(XEN)
(XEN) *** Dumping CPU1 host state: ***
(XEN) ----[ Xen-4.8.5 arm64 debug=n Tainted: C ]----
(XEN) CPU: 1
(XEN) PC: 0000000000243ad8 idle_loop+0x74/0x11c
(XEN) LR: 0000000000243ae0
(XEN) SP: 00008000ff1bfe70
(XEN) CPSR: 20000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN) X0: 0000000000000000 X1: 00008000feeb8680 X2: 0000000000000001
(XEN) X3: fffffffffffffed4 X4: 00008000feeb8680 X5: 0000000000000000
(XEN) X6: 00008000ff16dc40 X7: 00008000ff16dc58 X8: 00008000ff1bfe08
(XEN) X9: 0000000000262458 X10: 000000000000000a X11: 00008000ff1bfbe9
(XEN) X12: 0000000000000031 X13: 0000000000000001 X14: 00008000ff1bfbe8
(XEN) X15: 0000000000000020 X16: 0000000000000000 X17: 0000000000000000
(XEN) X18: 0000000000000000 X19: 0000000000302448 X20: 0000000000308d18
(XEN) X21: 00000000002cbf80 X22: 0000000000308d18 X23: 0000000000308d18
(XEN) X24: 0000000000000001 X25: 0000000000000001 X26: 0000000000000000
(XEN) X27: 0000000000000000 X28: 0000000000000000 FP: 00008000ff1bfe70
(XEN)
(XEN) VTCR_EL2: 80043594
(XEN) VTTBR_EL2: 0000000000000000
(XEN)
(XEN) SCTLR_EL2: 30cd183d
(XEN) HCR_EL2: 000000000038663f
(XEN) TTBR0_EL2: 00000000fecfc000
(XEN)
(XEN) ESR_EL2: 00000000
(XEN) HPFAR_EL2: 0000000000000000
(XEN) FAR_EL2: 0000000000000000
(XEN)
(XEN) Xen stack trace from sp=00008000ff1bfe70:
(XEN) 00008000ff1bfe80 0000000000250f3c 0000000000308d18 0000000000000001
(XEN) 0000000000000001 0000000000000001 0000000000400000 0900494931070860
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN) [<0000000000243ad8>] idle_loop+0x74/0x11c (PC)
(XEN) [<0000000000243ae0>] idle_loop+0x7c/0x11c (LR)
(XEN) [<0000000000250f3c>] start_secondary+0xfc/0x10c
(XEN) [<0000000000000001>] 0000000000000001
(XEN)
(XEN) *** Dumping CPU2 host state: ***
(XEN) ----[ Xen-4.8.5 arm64 debug=n Tainted: C ]----
(XEN) CPU: 2
(XEN) PC: 0000000000243ad8 idle_loop+0x74/0x11c
(XEN) LR: 0000000000243ae0
(XEN) SP: 00008000ff1afe70
(XEN) CPSR: 20000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN) X0: 0000000000000000 X1: 00008000feeae680 X2: 0000000000000002
(XEN) X3: fffffffffffffed4 X4: 00008000feeae680 X5: 0000000000000000
(XEN) X6: 00008000ff16df20 X7: 00008000ff16df38 X8: 00008000ff1afe08
(XEN) X9: 0000000000262458 X10: 000000000000000a X11: 00008000ff1afbe9
(XEN) X12: 0000000000000032 X13: 0000000000000001 X14: 00008000ff1afbe8
(XEN) X15: 0000000000000020 X16: 0000000000000000 X17: 0000000000000000
(XEN) X18: 0000000000000000 X19: 0000000000302448 X20: 0000000000308d18
(XEN) X21: 00000000002cbf80 X22: 0000000000308d18 X23: 0000000000308d18
(XEN) X24: 0000000000000002 X25: 0000000000000001 X26: 0000000000000000
(XEN) X27: 0000000000000000 X28: 0000000000000000 FP: 00008000ff1afe70
(XEN)
(XEN) VTCR_EL2: 80043594
(XEN) VTTBR_EL2: 0000000000000000
(XEN)
(XEN) SCTLR_EL2: 30cd183d
(XEN) HCR_EL2: 000000000038663f
(XEN) TTBR0_EL2: 00000000fecfc000
(XEN)
(XEN) ESR_EL2: 00000000
(XEN) HPFAR_EL2: 0000000000000000
(XEN) FAR_EL2: 0000000000000000
(XEN)
(XEN) Xen stack trace from sp=00008000ff1afe70:
(XEN) 00008000ff1afe80 0000000000250f3c 0000000000308d18 0000000000000002
(XEN) 0000000000000002 0000000000000001 0000000000400000 6002c108894108ca
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN) [<0000000000243ad8>] idle_loop+0x74/0x11c (PC)
(XEN) [<0000000000243ae0>] idle_loop+0x7c/0x11c (LR)
(XEN) [<0000000000250f3c>] start_secondary+0xfc/0x10c
(XEN) [<0000000000000002>] 0000000000000002
(XEN)
(XEN) *** Dumping CPU3 host state: ***
(XEN) ----[ Xen-4.8.5 arm64 debug=n Tainted: C ]----
(XEN) CPU: 3
(XEN) PC: 0000000000243ad8 idle_loop+0x74/0x11c
(XEN) LR: 0000000000243ae0
(XEN) SP: 00008000ff1a7e70
(XEN) CPSR: 20000249 MODE:64-bit EL2h (Hypervisor, handler)
(XEN) X0: 0000000000000000 X1: 00008000feeaa680 X2: 0000000000000003
(XEN) X3: fffffffffffffed4 X4: 00008000feeaa680 X5: 0000000000000000
(XEN) X6: 00008000ff1b4180 X7: 00008000ff1b4198 X8: 00008000ff1a7e08
(XEN) X9: 0000000000262458 X10: 000000000000000a X11: 00008000ff1a7be9
(XEN) X12: 0000000000000033 X13: 0000000000000001 X14: 00008000ff1a7be8
(XEN) X15: 0000000000000020 X16: 0000000000000000 X17: 0000000000000000
(XEN) X18: 0000000000000000 X19: 0000000000302448 X20: 0000000000308d18
(XEN) X21: 00000000002cbf80 X22: 0000000000308d18 X23: 0000000000308d18
(XEN) X24: 0000000000000003 X25: 0000000000000001 X26: 0000000000000000
(XEN) X27: 0000000000000000 X28: 0000000000000000 FP: 00008000ff1a7e70
(XEN)
(XEN) VTCR_EL2: 80043594
(XEN) VTTBR_EL2: 0000000000000000
(XEN)
(XEN) SCTLR_EL2: 30cd183d
(XEN) HCR_EL2: 000000000038663f
(XEN) TTBR0_EL2: 00000000fecfc000
(XEN)
(XEN) ESR_EL2: 00000000
(XEN) HPFAR_EL2: 0000000000000000
(XEN) FAR_EL2: 0000000000000000
(XEN)
(XEN) Xen stack trace from sp=00008000ff1a7e70:
(XEN) 00008000ff1a7e80 0000000000250f3c 0000000000308d18 0000000000000003
(XEN) 0000000000000003 0000000000000001 0000000000400000 70c821138b0c9de0
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN) [<0000000000243ad8>] idle_loop+0x74/0x11c (PC)
(XEN) [<0000000000243ae0>] idle_loop+0x7c/0x11c (LR)
(XEN) [<0000000000250f3c>] start_secondary+0xfc/0x10c
(XEN) [<0000000000000003>] 0000000000000003
(XEN)
(XEN)
It seems the DOM0 kernel did not get added to the task list….
Boot args for Xen and Dom0 are here:
(XEN) Checking for initrd in /chosen
(XEN) linux,initrd limits invalid: 0000000084100000 >= 0000000084100000
(XEN) RAM: 0000000080000000 - 00000000fedfffff
(XEN) RAM: 0000000100000000 - 000000017f1fffff
(XEN)
(XEN) MODULE[0]: 00000000fc7f8000 - 00000000fc82d000 Device Tree
(XEN) MODULE[1]: 00000000e1000000 - 00000000e31bc808 Kernel console=hvc0 earlyprintk=xen earlycon=xen rootfstype=ext4 rw rootwait root=/dev/mmcblk0p1 rdinit=/sbin/init
(XEN) RESVD[0]: 0000000080000000 - 0000000080020000
(XEN) RESVD[1]: 00000000e3500000 - 00000000e3535000
(XEN) RESVD[2]: 00000000fc7f8000 - 00000000fc82d000
(XEN)
(XEN) Command line: console=dtuart earlyprintk=xen earlycon=uart8250,mmio32,0x70006000 sync_console dom0_mem=512M log_lvl=all guest_loglvl=all console_to_ring
(XEN) Placing Xen at 0x00000000fec00000-0x00000000fee00000
Thanks,
Srini
From: Srinivas Bangalore <srini@xxxxxxxxxx>
Sent: Monday, July 20, 2020 3:51 AM
To: 'xen-devel@xxxxxxxxxxxxxxxxxxxx' <xen-devel@xxxxxxxxxxxxxxxxxxxx>
Subject: Porting Xen to Jetson Nano
Hello,
I am trying to get Xen working on a Jetson Nano board (which is based on NVIDIA’s Tegra210 SoC). After some searching through the Xen-devel archives, I learnt that there was a set of patches developed in 2017 to port Xen to Tegra (https://lists.xenproject.org/archives/html/xen-devel/2017-04/msg00991.html). However these patches don’t appear in the main source repository. Therefore, I applied these manually to Xen-4.8.5. With these changes, Xen now boots up successfully on the Jetson Nano, but there is no Dom0 output on the console. I can switch between Xen and Dom0 with CTRL-a-a-a.
I am using Linux kernel version 5.7 for Dom0. I also tried using the native Linux kernel that comes with the Nano board, but that doesn’t help.
Here’s the console screen capture:
## Flattened Device Tree blob at e3000000
Booting using the fdt blob at 0xe3000000
reserving fdt memory region: addr=80000000 size=20000
reserving fdt memory region: addr=e3000000 size=35000
Loading Device Tree to 00000000fc7f8000, end 00000000fc82ffff ... OK
Starting kernel ...
- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) linux,initrd limits invalid: 0000000084100000 >= 0000000084100000
(XEN) RAM: 0000000080000000 - 00000000fedfffff
(XEN) RAM: 0000000100000000 - 000000017f1fffff
(XEN)
(XEN) MODULE[0]: 00000000fc7f8000 - 00000000fc82d000 Device Tree
(XEN) MODULE[1]: 00000000e1000000 - 00000000e2cbe200 Kernel console=hvc0 earlyprintk=uart8250-32bit,0x70006000 rootfstype=ext4 rw rootwait root=/dev/mmcblk0p1
(XEN) RESVD[0]: 0000000080000000 - 0000000080020000
(XEN) RESVD[1]: 00000000e3000000 - 00000000e3035000
(XEN) RESVD[2]: 00000000fc7f8000 - 00000000fc82d000
(XEN)
(XEN) Command line: console=dtuart earlyprintk=xen earlycon=xenboot dom0_mem=512M loglevel=all
(XEN) Placing Xen at 0x00000000fec00000-0x00000000fee00000
(XEN) Update BOOTMOD_XEN from 0000000080080000-0000000080188e01 => 00000000fec00000-00000000fed08e01
(XEN) Domain heap initialised
(XEN) Taking dtuart configuration from /chosen/stdout-path
(XEN) Looking for dtuart at "/serial@70 Xen 4.8.5
(XEN) Xen version 4.8.5 (srinivas@) (aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0) debug=n Sun Jul 19 07:44:00 PDT 2020
(XEN) Latest ChangeSet:
(XEN) Processor: 411fd071: "ARM Limited", variant: 0x1, part 0xd07, rev 0x1
(XEN) 64-bit Execution:
(XEN) Processor Features: 0000000000002222 0000000000000000
(XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN) Extensions: FloatingPoint AdvancedSIMD
(XEN) Debug Features: 0000000010305106 0000000000000000
(XEN) Auxiliary Features: 0000000000000000 0000000000000000
(XEN) Memory Model Features: 0000000000001124 0000000000000000
(XEN) ISA Features: 0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN) Processor Features: 00000131:00011011
(XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN) Extensions: GenericTimer Security
(XEN) Debug Features: 03010066
(XEN) Auxiliary Features: 00000000
(XEN) Memory Model Features: 10101105 40000000 01260000 02102211
(XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 19200 KHz
(XEN) GICv2 initialization:
(XEN) gic_dist_addr=0000000050041000
(XEN) gic_cpu_addr=0000000050042000
(XEN) gic_hyp_addr=0000000050044000
(XEN) gic_vcpu_addr=0000000050046000
(XEN) gic_maintenance_irq=25
(XEN) GICv2: 224 lines, 4 cpus, secure (IID 0200143b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) Bringing up CPU1
- CPU 00000001 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Bringing up CPU2
- CPU 00000002 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Bringing up CPU3
- CPU 00000003 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Brought up 4 CPUs
(XEN) P2M: 44-bit IPA with 44-bit PA
(XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
(XEN) I/O virtualisation disabled
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 00000000e1000000
(XEN) Allocating 1:1 mappings totalling 512MB for dom0:
(XEN) BANK[0] 0x000000a0000000-0x000000c0000000 (512MB)
(XEN) Grant table range: 0x000000fec00000-0x000000fec60000
(XEN) Loading zImage from 00000000e1000000 to 00000000a0080000-00000000a1d3e200
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading dom0 DTB to 0x00000000a8000000-0x00000000a8034354
(XEN) Scrubbing Free RAM on 1 nodes using 4 CPUs
(XEN) ........done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 300kB init memory.
Any suggestions/pointers to move forward would be much appreciated.
Thanks,
Srini
