[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Minios-devel] Unikraft: _init_mem_clear_bootstrap() Fix
On Feb 1, 2018, at 2:57 PM, Simon Kuenzer <simon.kuenzer@xxxxxxxxx> wrote:
Hi Bruno,
Hi Simon,
thanks a lot for investigating this issue!
I had a similar issue with Xen on ARM where the zImage header was sometimes placed to a region that was out of range for the loader. I did not find the time yet to figure out why exactly this sometimes happens. I guess it has to do with the compiler flags -fdata-sections -ffunction-sections and --gc-sections for the linker. Any idea?
I have been reading on some sites about this and looks like those flags might make it work (they were mentioned in many discussions). Unfortunately, I am not that familiar with all those flags but I will try to understand with greater detail how they are used.
Could you send me your configuration (.config)? I wonder if I could reproduce the mis-placement that you saw; for now I couldn’t:
Here is the .config file I am using:
#
# Automatically generated file; DO NOT EDIT.
# Unikraft/0.2~8b94640-custom Configuration
#
#
# Architecture Selection
#
ARCH_X86_64=y
# ARCH_ARM_32 is not set
# MARCH_NATIVE is not set
MARCH_GENERIC=y
# MARCH_NOCONA is not set
# MARCH_CORE2 is not set
# MARCH_COREI7 is not set
# MARCH_COREI7AVX is not set
# MARCH_COREI7AVXI is not set
# MARCH_ATOM is not set
# MARCH_K8 is not set
# MARCH_K8SSE3 is not set
# MARCH_AMDFAM10 is not set
# MARCH_BTVER1 is not set
# MARCH_BDVER1 is not set
# MARCH_BDVER2 is not set
# MARCH_BDVER3 is not set
# MARCH_BTVER2 is not set
#
# Platform Support
#
PLAT_XEN=y
XEN_DBGEMERGENCY=y
# PLAT_KVM is not set
# PLAT_LINUXU is not set
#
# Library Configuration
#
HAVE_BOOTENTRY=y
# HAVE_LIBC is not set
HAVE_SCHED=y
LIBUKBOOT=y
LIBUKBOOT_BANNER=y
LIBUKBOOT_MAXNBARGS=60
LIBUKBOOT_INITALLOC=y
LIBUKDEBUG=y
LIBUKDEBUG_PRINTK=y
LIBUKDEBUG_PRINTD=y
# LIBUKDEBUG_PRINTD_EXTRA is not set
# LIBUKDEBUG_PRINTD_INFO is not set
# LIBUKDEBUG_PRINTD_WARN is not set
LIBUKDEBUG_PRINTD_ERR=y
# LIBUKDEBUG_PRINTD_CRIT is not set
LIBUKDEBUG_NOREDIR=y
# LIBUKDEBUG_REDIR_PRINTD is not set
# LIBUKDEBUG_REDIR_PRINTK is not set
LIBUKDEBUG_ENABLE_ASSERT=y
LIBUKARGPARSE=y
LIBNOLIBC=y
LIBUKALLOC=y
LIBUKALLOC_IFPAGES=y
# LIBUKALLOC_IFSTATS is not set
LIBUKALLOCBBUDDY=y
LIBUKSCHED=y
LIBUKSCHEDCOOP=y
# LIBFDT is not set
#
# Build Options
#
# OPTIMIZE_NONE is not set
OPTIMIZE_PERF=y
# OPTIMIZE_SIZE is not set
#
# Hint: Specify a CPU type to get most benefits from performance optimization
#
# OPTIMIZE_DEADELIM is not set
DEBUG_SYMBOLS=y
# DEBUG_SYMBOLS_LVL1 is not set
# DEBUG_SYMBOLS_LVL2 is not set
DEBUG_SYMBOLS_LVL3=y
OPTIMIZE_DBGFILE=y
OPTIMIZE_SYMFILE=y
OPTIMIZE_STRIP=y
# RECORD_BUILDTIME is not set
CROSS_COMPILE=""
#
# Application Options
#
APPHELLOWORLD_DEPENDENCIES=y
APPHELLOWORLD_PRINTARGS=y
UK_NAME=“helloworld"
nm build/helloworld_xen-x86_64.dbg | sort
0000000000000000 T _libxenplat_start
0000000000000000 T _text
0000000000000008 a RSP_OFFSET
0000000000000017 t stack_start
00000000000000fc a KERNEL_CS_MASK
0000000000001000 t _libxenplat_shared_info
0000000000002000 t hypercall_page
0000000000003000 t error_entry
0000000000003069 t hypervisor_callback
0000000000003078 t hypervisor_callback2
00000000000030bb t error_exit
...
On 01.02.2018 07:35, Bruno Alvisio wrote:
Hello all,
Currently, the call to the function unikraft/plat/x86/mm.c:_init_mem_clear_bootstrap() from unikraft/plat/xen/x86/setup.c:_init_mem() causes unikraft to crash. I investigated this issue and noticed that the VA address of _libxenplat_start (kernel entry point) is located at address 0x1000 instead of 0x0. (The .text section starts at address 0x0 though) See below:
*balvisio@debianxen**:~/unikraft/apps/helloworld/build$ nm helloworld_xen-x86_64.dbg | grep start | sort*
*0000000000001000 T _libxenplat_start***
*0000000000001017 t stack_start*
*000000000000435b t thread_starter*
*000000000000a0a0 t schedcoop_thread_start*
*000000000000c00c d nonopt_start*
*000000000000c01c D __bss_start*
*0000000000057060 B HYPERVISOR_start_info***
I forced the linker script to position _libxenplat_start() at location 0x0 with the following patch and it seems to solve the problem:
*diff --git a/plat/xen/x86/entry64.S b/plat/xen/x86/entry64.S*
*index bdaa05a..94a5546 100644*
*--- a/plat/xen/x86/entry64.S*
*+++ b/plat/xen/x86/entry64.S*
@@ -51,7 +51,7 @@ ELFNOTE(Xen, XEN_ELFNOTE_XEN_VERSION, .asciz "xen-3.0")
.text
.globl _libxenplat_start
-
+.section .text.prologue, "ax"
_libxenplat_start:
#else
*diff --git a/plat/xen/x86/link64.ld b/plat/xen/x86/link64.ld*
*index 2842a75..837c500 100644*
*--- a/plat/xen/x86/link64.ld*
*+++ b/plat/xen/x86/link64.ld*
@@ -32,6 +32,7 @@ SECTIONS
. = 0x0;
_text = .;/* Text and read-only data */
.text : {
+*(.text.prologue)
*(.text)
*(.gnu.warning)
} = 0x2000
*diff --git a/plat/xen/x86/setup.c b/plat/xen/x86/setup.c*
*index cf8bad2..18fcd24 100644*
*--- a/plat/xen/x86/setup.c*
*+++ b/plat/xen/x86/setup.c*
@@ -156,7 +156,7 @@ static inline void _init_mem(void)
uk_printd(DLVL_INFO, "max_pfn: %lx\n", max_pfn);
_init_mem_build_pagetable(&start_pfn, &max_pfn);
-//_init_mem_clear_bootstrap(); /* FIXME - stack or text screwed up? */
+_init_mem_clear_bootstrap(); /* FIXME - stack or text screwed up? */
//_init_mem_set_readonly(&_text, &_erodata); /* FIXME - shared info ro? */
/* Fill out mrd array */
I am not sure if the problem can be solved by adding certain flag to the linker so that it automatically places _libxenplat_start() function at 0x0.
Any comments on this would be greatly appreciated and if this looks like a right solution I can send the PATCH to the list.
What happens actually to the other boot-critical symbols when you apply your patch (e.g., _libxenplat_shared_info, hypercall_page). For now the boot code expects them to be at 0x1000 and 0x2000.
Looks like everything is shifted by 0x1000: (they end up at 0x2000 and 0x3000 instead)
balvisio@debianxen:~/unikraft/apps/helloworld/build$ nm helloworld_xen-x86_64.dbg | grep shared_info | sort
0000000000002000 t _libxenplat_shared_info
0000000000057078 B HYPERVISOR_shared_info
balvisio@debianxen:~/unikraft/apps/helloworld/build$ nm helloworld_xen-x86_64.dbg | grep hypercall | sort
0000000000003000 t hypercall_page
I send you the whole map in case you want to take a look:
balvisio@debianxen:~/unikraft/apps/helloworld/build$ nm helloworld_xen-x86_64.dbg | sort
0000000000000000 t do_hypervisor_callback
0000000000000000 T _text
0000000000000008 a RSP_OFFSET
00000000000000c0 t ukplat_lcpu_irqs_handle_pending
00000000000000fc a KERNEL_CS_MASK
0000000000000100 t ukplat_memregion_count
0000000000000110 t ukplat_memregion_get
00000000000002c0 t _libxenplat_x86entry
0000000000000420 t do_stack_walk
0000000000000470 t dump_mem.part.0
00000000000004f0 t dump_regs
00000000000005d0 t do_trap
0000000000000610 t do_divide_error
0000000000000630 t do_int3
0000000000000650 t do_overflow
0000000000000670 t do_bounds
0000000000000690 t do_invalid_op
00000000000006b0 t do_device_not_available
00000000000006d0 t do_coprocessor_segment_overrun
00000000000006f0 t do_invalid_TSS
0000000000000710 t do_segment_not_present
0000000000000730 t do_stack_segment
0000000000000750 t do_alignment_check
0000000000000770 t do_machine_check
0000000000000790 t stack_walk
00000000000007a0 t do_page_fault
00000000000008b0 t do_general_protection
0000000000000940 t do_debug
0000000000000980 t do_coprocessor_error
00000000000009b0 t simd_math_error
00000000000009d0 t do_simd_coprocessor_error
00000000000009f0 t do_spurious_interrupt_bug
0000000000000a00 t trap_init
0000000000000a30 t trap_fini
0000000000001000 T _libxenplat_start
0000000000001017 t stack_start
0000000000002000 t _libxenplat_shared_info
0000000000003000 t hypercall_page
0000000000004000 t error_entry
0000000000004069 t hypervisor_callback
0000000000004078 t hypervisor_callback2
00000000000040bb t error_exit
000000000000414a t restore_all_enable_events
00000000000041a8 t scrit
00000000000041b2 t restore_end
00000000000041d9 t ecrit
00000000000041d9 t hypervisor_prologue
00000000000041e1 t critical_region_fixup
0000000000004230 t failsafe_callback
0000000000004235 t coprocessor_error
0000000000004247 t simd_coprocessor_error
0000000000004259 t device_not_available
000000000000426b t debug
000000000000427d t int3
000000000000428f t overflow
00000000000042a1 t bounds
00000000000042b3 t invalid_op
00000000000042c5 t coprocessor_segment_overrun
00000000000042d7 t invalid_TSS
00000000000042e7 t segment_not_present
00000000000042f7 t stack_segment
0000000000004307 t general_protection
0000000000004317 t alignment_check
0000000000004327 t divide_error
0000000000004339 t spurious_interrupt_bug
000000000000434b t page_fault
000000000000435b t thread_starter
0000000000004369 t __arch_switch_threads
0000000000004390 t new_pt_frame
00000000000045a0 t _init_mem_build_pagetable
0000000000004850 t _init_mem_set_readonly
0000000000004a20 t _init_mem_clear_bootstrap
0000000000004ac0 t _init_mem_prepare
0000000000004b00 t arch_init_events
0000000000004b40 t arch_unbind_ports
0000000000004b50 t arch_fini_events
0000000000004b70 t ukplat_monotonic_clock
0000000000004c70 t timer_handler
0000000000004c90 t block_domain
0000000000004d20 t ukplat_time_init
0000000000004d50 t ukplat_time_fini
0000000000004d70 t arch_thread_init
0000000000004e20 t arch_run_idle_thread
0000000000004e30 t ukplat_lcpu_save_irqf
0000000000004e50 t ukplat_lcpu_restore_irqf
0000000000004e80 t ukplat_lcpu_irqs_disabled
0000000000004ea0 t ukplat_lcpu_halt_to
0000000000004f00 t ukplat_lcpu_halt
0000000000004f10 t hvconsole_input
0000000000004f20 t _libxenplat_prepare_console
0000000000004f60 t _libxenplat_init_console
0000000000005020 t ukplat_coutd
0000000000005060 t ukplat_coutk
00000000000051c0 t ukplat_cink
00000000000051d0 t ukplat_terminate
0000000000005210 t ukplat_suspend
0000000000005220 t default_handler
0000000000005240 t do_event
00000000000052c0 t bind_evtchn
0000000000005350 t unbind_evtchn
0000000000005420 t unbind_all_ports
00000000000054c0 t bind_virq
0000000000005520 t bind_pirq
0000000000005580 t init_events
00000000000055e0 t fini_events
0000000000005600 t evtchn_alloc_unbound
0000000000005670 t evtchn_bind_interdomain
00000000000056e0 t evtchn_get_peercontext
0000000000005740 t mask_evtchn
0000000000005770 t unmask_evtchn
00000000000057f0 t clear_evtchn
0000000000005820 t ukplat_thread_ctx_init
0000000000005830 t ukplat_thread_ctx_switch
0000000000005840 t ukplat_thread_ctx_current
0000000000005850 t ukplat_thread_ctx_run_idle
0000000000005860 t exit_thread
0000000000005900 t ukplat_entry
0000000000005aa0 t ukplat_entry_argp
0000000000005b30 t main_thread_func
0000000000005b50 t uk_vprintk
0000000000005b90 t uk_printk
0000000000005c50 t _uk_vprintd
0000000000005e90 t _uk_printd
0000000000006120 t _hxd_outf
0000000000006260 t _hxd
0000000000006770 t uk_hexdumpsn
0000000000006800 t uk_hexdumpf
0000000000006880 t uk_hexdumpk
00000000000068e0 t _uk_hexdumpd
0000000000006930 t uk_argnparse
0000000000006a90 t _nolibc_errno
0000000000006aa0 t vsnprintf
0000000000007720 t vsprintf
0000000000007740 t snprintf
00000000000077c0 t sprintf
0000000000007850 t vfprintf
00000000000078c0 t fprintf
0000000000007950 t vprintf
0000000000007970 t printf
0000000000007a10 t fflush
0000000000007a20 t strtoul
0000000000007c40 t strtoll
0000000000007f20 t strtoull
0000000000008150 t memcpy
0000000000008180 t memset
00000000000081b0 t memchr
00000000000081f0 t memmove
0000000000008250 t memcmp
00000000000082a0 t strlen
00000000000082d0 t strnlen
0000000000008320 t strncpy
00000000000083b0 t strcpy
00000000000083d0 t strchr
0000000000008420 t strncmp
0000000000008470 t permute_args
0000000000008530 t parse_long_options
00000000000087d0 t getopt_internal
0000000000008d50 t getopt
0000000000008d70 t getopt_long
0000000000008d80 t getopt_long_only
0000000000008d90 t malloc
0000000000008de0 t calloc
0000000000008e30 t realloc
0000000000008e80 t posix_memalign
0000000000008ee0 t memalign
0000000000008f20 t free
0000000000008f80 t uk_alloc_register
0000000000008fd0 t uk_alloc_get_default
0000000000008fe0 t uk_alloc_set_default
0000000000009040 t uk_malloc_ifpages
00000000000090e0 t uk_free_ifpages
0000000000009150 t uk_realloc_ifpages
0000000000009230 t uk_posix_memalign_ifpages
0000000000009330 t uk_calloc_compat
00000000000093a0 t uk_memalign_compat
0000000000009410 t map_free
0000000000009560 t bbuddy_addmem
0000000000009720 t bbuddy_palloc
00000000000099a0 t bbuddy_pfree
0000000000009bb0 t uk_allocbbuddy_init
0000000000009d60 t uk_sched_register
0000000000009db0 t uk_sched_get_default
0000000000009dc0 t uk_sched_set_default
0000000000009e20 t uk_sched_thread_create
0000000000009ee0 t uk_sched_thread_destroy
0000000000009f50 t uk_sched_sleep
0000000000009fc0 t uk_thread_block_millis
0000000000009ff0 t uk_thread_block
000000000000a010 t uk_thread_wake
000000000000a030 t idle_thread_fn
000000000000a0a0 t schedcoop_thread_start
000000000000a0e0 t schedcoop_run
000000000000a100 t schedcoop_schedule
000000000000a2d0 t schedcoop_thread_stop
000000000000a360 t uk_schedcoop_init
000000000000a4bd T _etext
000000000000a4bd T _rodata
000000000000a4c0 t main
000000000000a5a0 r pt_prot
000000000000a5c0 r __func__.3082
000000000000a740 r hex2ascii_data
000000000000a780 r _nolibc_ctype
000000000000a880 r __func__.2126
000000000000c000 D _data
000000000000c000 D _erodata
000000000000c000 d newline.1910
000000000000c004 d prevlvl.1911
000000000000c008 d nonopt_end
000000000000c00c d nonopt_start
000000000000c010 d optopt
000000000000c014 d optind
000000000000c018 d opterr
000000000000c01c D __bss_start
000000000000c01c D _edata
000000000000c020 d trap_table
000000000000c140 d stderr
000000000000c148 d stdout
000000000000c150 d stdin
000000000000c158 d place
000000000000c160 b _GLOBAL_OFFSET_TABLE_
000000000000c180 b cmdline
000000000000c580 b handling_pg_fault
000000000000c5a0 b mmu_updates.3503
000000000000e5c0 b mmu_updates.3486
00000000000105e0 b cpu0_pda
0000000000010600 b irqstack
0000000000030600 b port
0000000000030620 b shadow
0000000000030640 b console_ready
0000000000030644 b console_evtchn
0000000000030648 b console_ring
0000000000030660 b bound_ports
00000000000306e0 b ev_actions
00000000000366e0 b __errno
00000000000366e8 b uk_alloc_head
0000000000037000 B pt_base
0000000000037008 B _libxenplat_mrd_num
0000000000037020 B _libxenplat_mrd
0000000000037050 B optarg
0000000000037060 B _libxenplat_bootstack
0000000000057060 B HYPERVISOR_start_info
0000000000057068 B in_callback
000000000005706c B optreset
0000000000057070 B uk_sched_head
0000000000057078 B HYPERVISOR_shared_info
0000000000057080 B mfn_zero
0000000000057088 B phys_to_machine_mapping
0000000000057090 B _end
0000000080000000 a NMI_MASK
Cheers >
Bruno
Cheers,
Simon
|
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel
|
