Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Julien Grall <julien@xxxxxxx>]

(+ Stefano)

On 25/08/2020 13:04, Charles Chiou wrote:
> Hello,
Hello,

> I'm having problem with virtual console when booting up 32 bit guests. I'm using qemu 
> virt machine, running xen 64 bit, DOM0 Linux is 64 bit. The 64 bit guests that I've tried 
> (Linux, an RTOS) have no issues, but when I tried to run 32 bit OS (tried linux, FreeRTOS 
> from Galois, a bare-metal app), I get "Invalid MFN 0x33a08" or similar messages 
> when making HVC calls such as:
>
>          mov     r0, #0
>          mov     r1, #35
>          ldr     r2, =banner
>          mov     r12, #18
>          hvc     #0xea1<----
>
> and get this at the HVC call to XEN:
>
> (XEN) p2m.c:1919: d8v0: Invalid MFN 0x33a08
>
> Not sure if there's misconfiguration somewhere, or compile options, etc. that 
> caused this... Any pointer is greatly appreciated!
>
> Below is output of similar problem when booting up a 32 bit linux kernel:
Looking at the code, this seems like an issue when trying to translate a 
guest virtual address to a machine address.
A few questions:
   - Which QEMU version are you using?
   - What's your Linux configuration? Are you using LPAE or short page 
tables?
I am also not sure what's the state of virtualization support in QEMU 
for 32-bit guest. I have CCed Stefano who may have a better idea.
Best regards,

> / # xl -v create -c /share/misc/linux32.cfg
> Parsing config from /share/misc/linux32.cfg
> libxl: info: libxl_create.c:122:libxl__domain_build_info_setdefault: qemu-xen 
> is unavailable, using qemu-xen-traditional instead: No such file or directory
> libxl: detail: libxl_create.c:623:libxl__domain_make: passthrough: disabled
> domainbuilder: detail: xc_dom_allocate: cmdline="rw root=/dev/ram rdinit=/sbin/init 
> earlyprintk=serial,ttyAMA0", features=""
> domainbuilder: detail: xc_dom_kernel_file: 
> filename="/share/linux-domu-32bit/arch/arm/boot/zImage"
> domainbuilder: detail: xc_dom_malloc_filemap    : 5904 kB
> domainbuilder: detail: xc_dom_module_file: filename="/share/dom0-rootfs.img.gz"
> domainbuilder: detail: xc_dom_malloc_filemap    : 13553 kB
> domainbuilder: detail: xc_dom_boot_xen_init: ver 4.15, caps xen-3.0-aarch64 
> xen-3.0-armv7l
> domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
> domainbuilder: detail: xc_dom_parse_image: called
> domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
> domainbuilder: detail: loader probe failed
> domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64) loader 
> ...
> domainbuilder: detail: xc_dom_probe_zimage64_kernel: kernel is not an arm64 
> Image
> domainbuilder: detail: loader probe failed
> domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM32) loader 
> ...
> domainbuilder: detail: loader probe OK
> domainbuilder: detail: xc_dom_parse_zimage32_kernel: called
> domainbuilder: detail: xc_dom_parse_zimage32_kernel: xen-3.0-armv7l: 0x40008000 
> -> 0x405cc200
> domainbuilder: detail: xc_dom_devicetree_mem: called
> domainbuilder: detail: xc_dom_mem_init: mem 128 MB, pages 0x8000 pages, 4k each
> domainbuilder: detail: xc_dom_mem_init: 0x8000 pages
> domainbuilder: detail: xc_dom_boot_mem_init: called
> domainbuilder: detail: set_mode: guest xen-3.0-armv7l, address size 32
> domainbuilder: detail: populate_guest_memory: populating RAM @ 
> 0000000040000000-0000000048000000 (128MB)
> domainbuilder: detail: populate_one_size: populated 0x40/0x40 entries with 
> shift 9
> domainbuilder: detail: meminit: placing boot modules at 0x472c2000
> domainbuilder: detail: meminit: ramdisk: 0x472c2000 -> 0x47fff000
> domainbuilder: detail: meminit: devicetree: 0x47fff000 -> 0x48000000
> domainbuilder: detail: xc_dom_build_image: called
> domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 
> 0x40008+0x5c5 at 0xffff93aea000
> domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0x40008000 -> 
> 0x405cd000  (pfn 0x40008 + 0x5c5 pages)
> domainbuilder: detail: xc_dom_load_zimage_kernel: called
> domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 
> 0x40008000-0x405cd000
> domainbuilder: detail: xc_dom_load_zimage_kernel: copy 6046208 bytes from blob 
> 0xffff94dec000 to dst 0xffff93aea000
> domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 
> 0x472c2+0xd3d at 0xffff92dad000
> domainbuilder: detail: xc_dom_alloc_segment:   module0      : 0x472c2000 -> 
> 0x47fff000  (pfn 0x472c2 + 0xd3d pages)
> domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 
> 0x47fff+0x1 at 0xffff95847000
> domainbuilder: detail: xc_dom_alloc_segment:   devicetree   : 0x47fff000 -> 
> 0x48000000  (pfn 0x47fff + 0x1 pages)
> domainbuilder: detail: alloc_magic_pages: called
> domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0x48000000
> domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0x0
> domainbuilder: detail: xc_dom_boot_image: called
> domainbuilder: detail: bootearly: doing nothing
> domainbuilder: detail: xc_dom_compat_check: supported guest type: 
> xen-3.0-aarch64
> domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-armv7l 
> <= matches
> domainbuilder: detail: start_info_arm: called
> domainbuilder: detail: domain builder memory footprint
> domainbuilder: detail:    allocated
> domainbuilder: detail:       malloc             : 117 kB
> domainbuilder: detail:       anon mmap          : 0 bytes
> domainbuilder: detail:    mapped
> domainbuilder: detail:       file mmap          : 19457 kB
> domainbuilder: detail:       domU mmap          : 19468 kB
> domainbuilder: detail: vcpu_arm32: called
> domainbuilder: detail: Initial state CPSR 0x1d3 PC 0x40008000
> domainbuilder: detail: compat_gnttab_hvm_seed: d7: pfn=0x38000
> domainbuilder: detail: xc_dom_set_gnttab_entry: d7 gnt[0] -> d0 0x39000
> domainbuilder: detail: xc_dom_set_gnttab_entry: d7 gnt[1] -> d0 0x39001
> domainbuilder: detail: xc_dom_release: called
> (XEN) p2m.c:1919: d7v0: Invalid MFN 0x3b401
> (XEN) d7v0: vGICR: SGI: unhandled word write 0x000000ffffffff to ICACTIVER0
> (XEN) p2m.c:1919: d7v0: Invalid MFN 0x3b401
> (XEN) p2m.c:1919: d7v0: Invalid MFN 0x39281
>
> --
> Charles Chiou
> NOTE: This email (including attachments) contain Ambarella Proprietary and/or 
> Confidential Information and is intended solely for the use of the 
> individual(s) to whom it is addressed. Any unauthorized review, use, 
> disclosure, distribute, copy, or print is prohibited. If you are not an 
> intended recipient, please contact the sender by reply email and destroy all 
> copies of the original message. Thank you.
--
Julien Grall