Re: [Xen-devel] [questions] Dom0/DomU on ARM under Xen

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Wed, 8 Aug 2012, Alexey Klimov wrote:
> 2012/8/1 Alexey Klimov <trashsee@xxxxxxxxx>:
> > And i saw that Ian set up git repository for xen with latest patches
> > for ARM. So i'll try to use this repository.
> 
> Hello Stefano and Ian,
> 
> I used new Ian xen-unstable git repository
> (git://xenbits.xen.org/people/ianc/xen-unstable.git arm-for-4.) and
> Stefano linux kernel git repository (
> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git
> 3.5-rc7-arm-2) with additional patches:
> 
> - for linux kernel "xen/events: fix unmask_evtchn for PV on HVM guests",
> - ARM hypercall ABI: 64 bit ready patch series for xen and attached
> few versions of xcbuild (early version of Ian and latest version).
> After applying 64-bit ready patches i observed such errors when
> building xen and tools:
> 
> 1)
> for i in public/callback.h public/dom0_ops.h public/elfnote.h
> public/event_channel.h public/features.h public/grant_table.h
> public/kexec.h public/mem_event.h public/memory.h public/nmi.h
> public/physdev.h public/platform.h public/sched.h public/tmem.h
> public/trace.h public/vcpu.h public/version.h public/xen-compat.h
> public/xen.h public/xencomm.h public/xenoprof.h public/hvm/e820.h
> public/hvm/hvm_info_table.h public/hvm/hvm_op.h public/hvm/ioreq.h
> public/hvm/params.h public/io/blkif.h public/io/console.h
> public/io/fbif.h public/io/fsif.h public/io/kbdif.h
> public/io/libxenvchan.h public/io/netif.h public/io/pciif.h
> public/io/protocols.h public/io/ring.h public/io/tpmif.h
> public/io/usbif.h public/io/vscsiif.h public/io/xenbus.h
> public/io/xs_wire.h; do gcc -ansi -include stdint.h -Wall -W -Werror
> -S -o /dev/null -xc $i || exit 1; echo $i; done >headers.chk.new
> public/version.h:61:5: error: unknown type name 'xen_ulong_t'
> make[3]: *** [headers.chk] Error 1
> make[3]: Leaving directory `/src/xen/xen/include'
> 
> Fixed by inserting #include "arch-arm.h" in xen/include/public/version.h

I think that this is a legitimate error, I wasn't seeing it because I am
cross-compiling.


> 2)
> building 'xc' extension
> gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -O1 -fno-omit-frame-pointer -marm -g
> -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> -Wdeclaration-after-statement -Wno-unused-but-set-variable
> -D__XEN_TOOLS__ -MMD -MF .build.d -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
> -fno-optimize-sibling-calls -fPIC -I../../tools/include
> -I../../tools/libxc -Ixen/lowlevel/xc -I/usr/include/python2.7 -c
> xen/lowlevel/xc/xc.c -o
> build/temp.linux-armv7l-2.7/xen/lowlevel/xc/xc.o -fno-strict-aliasing
> -Werror
> xen/lowlevel/xc/xc.c: In function 'pyxc_xeninfo':
> xen/lowlevel/xc/xc.c:1442:5: error: format '%lx' expects argument of
> type 'long unsigned int', but argument 4 has type 'xen_ulong_t'
> [-Werror=format]
> xen/lowlevel/xc/xc.c:1442:5: error: format '%lx' expects argument of
> type 'long unsigned int', but argument 4 has type 'xen_ulong_t'
> [-Werror=format]
> cc1: all warnings being treated as errors
> 
> Just commented snprintf(str, sizeof(str), "virt_start=0x%lx",
> p_parms.virt_start); in xc.c

That is also another legitimate error, I'll fix it in the next version
of the patch series. Thanks for testing!


> Then it compiled and i tried to run DomU. It looks like allocation
> console_pfn and xenstore_pfn in alloc_magic_pages() in xc_dom_arm.c
> creates real pain for me. With this allocation/patch xen prints "bad
> p2m lookup" messages before booting DomU
> (XEN) bad p2m lookup
> (XEN) dom1 IPA 0x0000000090000000
> (XEN) P2M @ 02ffcac0 mfn:0xffe56
> (XEN) 1ST[0x2] = 0x00000000f3f686ff
> (XEN) 2ND[0x80] = 0x0000000000000000
> (XEN) bad p2m lookup
> (XEN) dom1 IPA 0x0000000090001000
> (XEN) P2M @ 02ffcac0 mfn:0xffe56
> (XEN) 1ST[0x2] = 0x00000000f3f686ff
> (XEN) 2ND[0x80] = 0x0000000000000000
> (XEN) bad p2m lookup
> (XEN) dom1 IPA 0x0000000090001000
> (XEN) P2M @ 02ffcac0 mfn:0xffe56
> (XEN) 1ST[0x2] = 0x00000000f3f686ff
> (XEN) 2ND[0x80] = 0x0000000000000000
> 
> and then everything  hangs with translation fault:
> 
> (XEN) DOM1: Grant tables using version 1 layout.
> (XEN) DOM1: Grant table initialized
> (XEN) DOM1: NET: Registered protocol family 16
> (XEN) Guest data abort: Translation fault at level 2
> (XEN)     gva=88808804
> (XEN)     gpa=0000000090001804
> (XEN)     size=2 sign=0 write=0 reg=2
> (XEN)     eat=0 cm=0 s1ptw=0 dfsc=6
> (XEN) dom1 IPA 0x0000000090001804
> (XEN) P2M @ 02ffcac0 mfn:0xffe56
> (XEN) 1ST[0x2] = 0x00000000f3f686ff
> (XEN) 2ND[0x80] = 0x0000000000000000
> 
> Detailed log is attached.
> Ok, i moved allocation for console and xenstore pages back in
> arch_setup_meminit() like in
> http://lists.xen.org/archives/html/xen-devel/2012-06/msg01340.html and
> then added kernel parameter keep_bootcon in DomU  device tree file and
> everything booted up to "unable to open an initial console" and unable
> to mount rootfs.

You are probably missing Ian's fix to alloc_magic_pages:

http://marc.info/?l=xen-devel&m=134398933530124


> I still didn't learn how to deal with xenstore, hvc0,
> xvda and how to boot from initramfs on ARM using xcbuild but i'll try
> to understand and learn this :) So may be it's good thing to
> investigate or take deep look why add_to_physmap failed in xcbuild and
> why there is bad p2m lookup in xen. Log is attached.
> 
> Do you have any difference between Dom0 .config and DomU .config? Did
> you just attach initrd using xc_dom_ramdisk_file() call in xcbuild?
> Any special configuration of xen console/xen store?

I am just using one config, attached.


> Well, i dont mean that i'm doing everything correctly but i tried to
> run it fixing/commenting as much as i can. Could you please help if
> you have time? I can test new changes, sent other useful info/logs.

Looking at the guest data abort that you are getting, I think that you
didn't update the dts and dtsi to the latest version. They are attached
to the 00/23 email "Introduce Xen support on ARM" for the linux kernel.

Attachment: config-linux
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel