Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Charles Chiou <cchiou@xxxxxxxxxxxxx>]

> (+ Stefano)
>
> On 25/08/2020 13:04, Charles Chiou wrote:
> > Hello,
>
> Hello,
>
> > I'm having problem with virtual console when booting up 32 bit guests. I'm
> using qemu virt machine, running xen 64 bit, DOM0 Linux is 64 bit. The 64 bit
> guests that I've tried (Linux, an RTOS) have no issues, but when I tried to 
> run
> 32 bit OS (tried linux, FreeRTOS from Galois, a bare-metal app), I get 
> "Invalid
> MFN 0x33a08" or similar messages when making HVC calls such as:
> >
> >          mov     r0, #0
> >          mov     r1, #35
> >          ldr     r2, =banner
> >          mov     r12, #18
> >          hvc     #0xea1<----
> >
> > and get this at the HVC call to XEN:
> >
> > (XEN) p2m.c:1919: d8v0: Invalid MFN 0x33a08
> >
> > Not sure if there's misconfiguration somewhere, or compile options, etc.
> that caused this... Any pointer is greatly appreciated!
> >
> > Below is output of similar problem when booting up a 32 bit linux kernel:
>
> Looking at the code, this seems like an issue when trying to translate a guest
> virtual address to a machine address.
>
> A few questions:
>     - Which QEMU version are you using?
>     - What's your Linux configuration? Are you using LPAE or short page 
> tables?

I am using default ubuntu package on bionic:

$ qemu-system-aarch64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

I just tried the stable-4.14 branch and this "Invalid MFN 0x..." doesn't 
happen. It was on the master branch where I encountered this problem.
However, I don't seem to get anything to call guest_printk() after making 
hypervisor_console_io call even for 64-bit guest in this version (worked on 
master). This is a separate question: could that be a XEN compile option 
problem as I see "debug=n" from xen register dump:
(XEN) ----[ Xen-4.14.1-pre  arm64  debug=n   Not tainted ]----

I'm ignoring 32-bit Linux for now and trying to boot bare metal apps that calls 
hypervisor_console_io() at the very beginning of the boot-up. As I'm trying to 
port embedded RTOS apps (both 32 and 64 bits) to run as guests.

Thank you!


> I am also not sure what's the state of virtualization support in QEMU for 32-
> bit guest. I have CCed Stefano who may have a better idea.
>
> Best regards,
>
> > / # xl -v create -c /share/misc/linux32.cfg Parsing config from
> > /share/misc/linux32.cfg
> > libxl: info: libxl_create.c:122:libxl__domain_build_info_setdefault:
> > qemu-xen is unavailable, using qemu-xen-traditional instead: No such
> > file or directory
> > libxl: detail: libxl_create.c:623:libxl__domain_make: passthrough:
> > disabled
> > domainbuilder: detail: xc_dom_allocate: cmdline="rw root=/dev/ram
> rdinit=/sbin/init earlyprintk=serial,ttyAMA0", features=""
> > domainbuilder: detail: xc_dom_kernel_file: filename="/share/linux-domu-
> 32bit/arch/arm/boot/zImage"
> > domainbuilder: detail: xc_dom_malloc_filemap    : 5904 kB
> > domainbuilder: detail: xc_dom_module_file: filename="/share/dom0-
> rootfs.img.gz"
> > domainbuilder: detail: xc_dom_malloc_filemap    : 13553 kB
> > domainbuilder: detail: xc_dom_boot_xen_init: ver 4.15, caps
> > xen-3.0-aarch64 xen-3.0-armv7l
> > domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
> > domainbuilder: detail: xc_dom_parse_image: called
> > domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary
> loader ...
> > domainbuilder: detail: loader probe failed
> > domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64)
> loader ...
> > domainbuilder: detail: xc_dom_probe_zimage64_kernel: kernel is not an
> > arm64 Image
> > domainbuilder: detail: loader probe failed
> > domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM32)
> loader ...
> > domainbuilder: detail: loader probe OK
> > domainbuilder: detail: xc_dom_parse_zimage32_kernel: called
> > domainbuilder: detail: xc_dom_parse_zimage32_kernel: xen-3.0-armv7l:
> > 0x40008000 -> 0x405cc200
> > domainbuilder: detail: xc_dom_devicetree_mem: called
> > domainbuilder: detail: xc_dom_mem_init: mem 128 MB, pages 0x8000
> > pages, 4k each
> > domainbuilder: detail: xc_dom_mem_init: 0x8000 pages
> > domainbuilder: detail: xc_dom_boot_mem_init: called
> > domainbuilder: detail: set_mode: guest xen-3.0-armv7l, address size 32
> > domainbuilder: detail: populate_guest_memory: populating RAM @
> > 0000000040000000-0000000048000000 (128MB)
> > domainbuilder: detail: populate_one_size: populated 0x40/0x40 entries
> > with shift 9
> > domainbuilder: detail: meminit: placing boot modules at 0x472c2000
> > domainbuilder: detail: meminit: ramdisk: 0x472c2000 -> 0x47fff000
> > domainbuilder: detail: meminit: devicetree: 0x47fff000 -> 0x48000000
> > domainbuilder: detail: xc_dom_build_image: called
> > domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn
> 0x40008+0x5c5 at 0xffff93aea000
> > domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0x40008000 ->
> 0x405cd000  (pfn 0x40008 + 0x5c5 pages)
> > domainbuilder: detail: xc_dom_load_zimage_kernel: called
> > domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg
> > 0x40008000-0x405cd000
> > domainbuilder: detail: xc_dom_load_zimage_kernel: copy 6046208 bytes
> > from blob 0xffff94dec000 to dst 0xffff93aea000
> > domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn
> 0x472c2+0xd3d at 0xffff92dad000
> > domainbuilder: detail: xc_dom_alloc_segment:   module0      : 0x472c2000 -
> > 0x47fff000  (pfn 0x472c2 + 0xd3d pages)
> > domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn
> 0x47fff+0x1 at 0xffff95847000
> > domainbuilder: detail: xc_dom_alloc_segment:   devicetree   : 0x47fff000 ->
> 0x48000000  (pfn 0x47fff + 0x1 pages)
> > domainbuilder: detail: alloc_magic_pages: called
> > domainbuilder: detail: xc_dom_build_image  : virt_alloc_end :
> > 0x48000000
> > domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0x0
> > domainbuilder: detail: xc_dom_boot_image: called
> > domainbuilder: detail: bootearly: doing nothing
> > domainbuilder: detail: xc_dom_compat_check: supported guest type:
> > xen-3.0-aarch64
> > domainbuilder: detail: xc_dom_compat_check: supported guest type:
> > xen-3.0-armv7l <= matches
> > domainbuilder: detail: start_info_arm: called
> > domainbuilder: detail: domain builder memory footprint
> > domainbuilder: detail:    allocated
> > domainbuilder: detail:       malloc             : 117 kB
> > domainbuilder: detail:       anon mmap          : 0 bytes
> > domainbuilder: detail:    mapped
> > domainbuilder: detail:       file mmap          : 19457 kB
> > domainbuilder: detail:       domU mmap          : 19468 kB
> > domainbuilder: detail: vcpu_arm32: called
> > domainbuilder: detail: Initial state CPSR 0x1d3 PC 0x40008000
> > domainbuilder: detail: compat_gnttab_hvm_seed: d7: pfn=0x38000
> > domainbuilder: detail: xc_dom_set_gnttab_entry: d7 gnt[0] -> d0
> > 0x39000
> > domainbuilder: detail: xc_dom_set_gnttab_entry: d7 gnt[1] -> d0
> > 0x39001
> > domainbuilder: detail: xc_dom_release: called
> > (XEN) p2m.c:1919: d7v0: Invalid MFN 0x3b401
> > (XEN) d7v0: vGICR: SGI: unhandled word write 0x000000ffffffff to
> > ICACTIVER0
> > (XEN) p2m.c:1919: d7v0: Invalid MFN 0x3b401
> > (XEN) p2m.c:1919: d7v0: Invalid MFN 0x39281
NOTE: This email (including attachments) contain Ambarella Proprietary and/or 
Confidential Information and is intended solely for the use of the 
individual(s) to whom it is addressed. Any unauthorized review, use, 
disclosure, distribute, copy, or print is prohibited. If you are not an 
intended recipient, please contact the sender by reply email and destroy all 
copies of the original message. Thank you.