Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Charles Chiou <cchiou@xxxxxxxxxxxxx>]

> >>>>> Looking at the code, this seems like an issue when trying to
> >>>>> translate a guest virtual address to a machine address.
> >>>>>
> >>>>> A few questions:
> >>>>>      - Which QEMU version are you using?
> >>>>>      - What's your Linux configuration? Are you using LPAE or
> >>>>> short page
> >>> tables?
> >>>>
> >>>> I am using default ubuntu package on bionic:
> >>>>
> >>>> $ qemu-system-aarch64 --version
> >>>> QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29)
> >>>> Copyright
> >>>> (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> >>>>
> >>>> I just tried the stable-4.14 branch and this "Invalid MFN 0x..."
> >>>> doesn't
> >>> happen. It was on the master branch where I encountered this problem.
> >>>> However, I don't seem to get anything to call guest_printk() after
> >>>> making
> >>> hypervisor_console_io call even for 64-bit guest in this version
> >>> (worked on master). This is a separate question: could that be a XEN
> >>> compile option problem as I see "debug=n" from xen register dump:
> >>>> (XEN) ----[ Xen-4.14.1-pre  arm64  debug=n   Not tainted ]----
> >>>
> >>> The call chain is:
> >>>
> >>> xen/drivers/char/console.c:do_console_io
> >>> xen/drivers/char/console.c:guest_console_write
> >>> xen/drivers/char/console.c:guest_printk
> >>>
> >>> You can enable debug through kconfig by doing "make menuconfig"
> >>> under xen/
> >>
> >> Thank you for pointing this out! After I enabled the verbose debug
> messages, the hvc #0xea1 call is now taking effect.
> >
> > Great!
> >
> >
> >> I'm seeing the previous problem of "invalid MFN" after that. It seems that
> the pointer 0x40000058 had been mapped to MFN 0x2be08. How do I debug
> or where locate the problem? I'm very new to XEN so it's no obvious yet to
> me from reading the source code how after guest_printk(), xen translate the
> addresses.
> >
> > The baremetal application code and the disassembly look correct. I
> > wonder if the issue is that the baremetal application is passing a
> > guest physical address when Xen expects a guest virtual address.
>
> Bear in mind that issuing an hypercall with MMU disabled is usually risky
> because of memory attribute mismatch between Xen and the Guest.
>
> Although, you would only see data corruption and not "invalid MFN".
>
> >
> > The string gets copied by:
> >
> > xen/drivers/char/console.c:guest_console_write
> >
> > calling:
> >
> > copy_from_guest -> copy_from_guest_offset -> raw_copy_from_guest
> >
> > eventually it goes to:
> >
> > xen/arch/arm/guestcopy.c:copy_guest
> > xen/arch/arm/guestcopy.c:translate_get_page
> >
> >
> > Looking at the code, translate_get_page is called with linear=true
> > write=false. linear=true causes translate_get_page to call
> > get_page_from_gva. I wonder if it is possible that get_page_from_gva
> > is not doing the right thing here.
>
> When the MMU is turned off, VA == PA. The AT instruction is able to deal
> with such situation. However...

Hi Stefano, thank you for the pointers. It was very helpful to get started in 
tracing the internals of xen.

>  >>>> QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29)
> Copyright  >>>> (c) 2003-2017 Fabrice Bellard and the QEMU Project
> developers
>
>
> ... this is fairly an old version of QEMU. I remember some issues with the
> implementation of the AT instruction in QEMU. Looking at the commit logs, it
> seems there are a few fixes that is not part of 2.11.1.
>
> I would highly recomend to use a more recent version of QEMU in order to
> narrow down the problem.

Hi Julien, switching qemu to v3.1.1.1 solved the problem! (Tried 5.x but ran 
into other problems, but those are something else). The address translation now 
seems ok now and gvirt_to_maddr() is returning the right values after executing 
"at s12e1w, %0", and guest console output is working. Thank you!

--
Charles Chiou
NOTE: This email (including attachments) contain Ambarella Proprietary and/or 
Confidential Information and is intended solely for the use of the 
individual(s) to whom it is addressed. Any unauthorized review, use, 
disclosure, distribute, copy, or print is prohibited. If you are not an 
intended recipient, please contact the sender by reply email and destroy all 
copies of the original message. Thank you.