Re: [PATCH 1/2] Add libfuzzer target to fuzz/x86_instruction_emulator

[Tamas K Lengyel <tamas@xxxxxxxxxxxxx>]

On Tue, Jun 25, 2024 at 9:15 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
>
> On 25.06.2024 14:40, Tamas K Lengyel wrote:
> > On Tue, Jun 25, 2024 at 7:52 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
> >>
> >> On 25.06.2024 13:12, Tamas K Lengyel wrote:
> >>> On Tue, Jun 25, 2024 at 2:00 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
> >>>>
> >>>> On 24.06.2024 23:23, Tamas K Lengyel wrote:
> >>>>> On Mon, Jun 24, 2024 at 11:55 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:
> >>>>>>
> >>>>>> On 21.06.2024 21:14, Tamas K Lengyel wrote:
> >>>>>>> @@ -58,6 +58,9 @@ afl-harness: afl-harness.o $(OBJS) cpuid.o 
> >>>>>>> wrappers.o
> >>>>>>>  afl-harness-cov: afl-harness-cov.o $(patsubst %.o,%-cov.o,$(OBJS)) 
> >>>>>>> cpuid.o wrappers.o
> >>>>>>>       $(CC) $(CFLAGS) $(GCOV_FLAGS) $(addprefix 
> >>>>>>> -Wl$(comma)--wrap=,$(WRAPPED)) $^ -o $@
> >>>>>>>
> >>>>>>> +libfuzzer-harness: $(OBJS) cpuid.o
> >>>>>>> +     $(CC) $(CFLAGS) $(LIB_FUZZING_ENGINE) -fsanitize=fuzzer $^ -o $@
> >>>>>>
> >>>>>> What is LIB_FUZZING_ENGINE? I don't think we have any use of that in 
> >>>>>> the
> >>>>>> tree anywhere.
> >>>>>
> >>>>> It's used by oss-fuzz, otherwise it's not doing anything.
> >>>>>
> >>>>>>
> >>>>>> I'm further surprised you get away here without wrappers.o.
> >>>>>
> >>>>> Wrappers.o was actually breaking the build for oss-fuzz at the linking
> >>>>> stage. It works just fine without it.
> >>>>
> >>>> I'm worried here, to be honest. The wrappers serve a pretty important
> >>>> role, and I'm having a hard time seeing why they shouldn't be needed
> >>>> here when they're needed both for the test and afl harnesses. Could
> >>>> you add some more detail on the build issues you encountered?
> >>>
> >>> With wrappers.o included doing the build in the oss-fuzz docker
> >>> (ubuntu 20.04 base) fails with:
> >>>
> >>> ...
> >>> clang -O1 -fno-omit-frame-pointer -gline-tables-only
> >>> -Wno-error=enum-constexpr-conversion
> >>> -Wno-error=incompatible-function-pointer-types
> >>> -Wno-error=int-conversion -Wno-error=deprecated-declarations
> >>> -Wno-error=implicit-function-declaration -Wno-error=implicit-int
> >>> -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address
> >>> -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link -m64
> >>> -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes
> >>> -Wno-unused-but-set-variable -Wno-unused-local-typedefs   -g3 -Werror
> >>> -Og -fno-omit-frame-pointer
> >>> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MP
> >>> -MF .libfuzzer-harness.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
> >>> -I/src/xen/tools/fuzz/x86_instruction_emulator/../../../tools/include
> >>> -D__XEN_TOOLS__ -iquote . -fsanitize=fuzzer -fsanitize=fuzzer
> >>> -Wl,--wrap=fwrite -Wl,--wrap=memcmp -Wl,--wrap=memcpy
> >>> -Wl,--wrap=memset -Wl,--wrap=printf -Wl,--wrap=putchar -Wl,--wrap=puts
> >>> -Wl,--wrap=snprintf -Wl,--wrap=strstr -Wl,--wrap=vprintf
> >>> -Wl,--wrap=vsnprintf fuzz-emul.o x86-emulate.o x86_emulate/0f01.o
> >>> x86_emulate/0fae.o x86_emulate/0fc7.o x86_emulate/decode.o
> >>> x86_emulate/fpu.o cpuid.o wrappers.o -o libfuzzer-harness
> >>> /usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 
> >>> 0x25
> >>> /usr/local/lib/clang/18/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer.a(fuzzer.o):
> >>> in function `std::__Fuzzer::__libcpp_snprintf_l(char*, unsigned long,
> >>> __locale_struct*, char const*, ...)':
> >>> cxa_noexception.cpp:(.text._ZNSt8__Fuzzer19__libcpp_snprintf_lEPcmP15__locale_structPKcz[_ZNSt8__Fuzzer19__libcpp_snprintf_lEPcmP15__locale_structPKcz]+0x9a):
> >>> undefined reference to `__wrap_vsnprintf'
> >>> clang: error: linker command failed with exit code 1 (use -v to see 
> >>> invocation)
> >>> make: *** [Makefile:62: libfuzzer-harness] Error 1
> >>> rm x86-emulate.c wrappers.c cpuid.c
> >>> make: Leaving directory '/src/xen/tools/fuzz/x86_instruction_emulator'
> >>> ERROR:__main__:Building fuzzers failed.
> >>
> >> Hmm, yes, means we'll need an actual vsnprintf() wrapper, not just a
> >> declaration thereof.
> >
> > I don't really get what this wrapper accomplishes
>
> They guard against clobbering of in-register state (SIMD registers in
> particular, but going forward maybe also eGRP-s as introduced by APX)
> by library functions called between emulation of individual insns (or,
> especially possible for fuzzing instrumented code, I think) even from
> in the middle of emulating an insn. (Something as simple as the
> compiler inserting a call to memcpy() or memset() somewhere in the
> translation of the emulator source code could also clobber state.)
>
> > and as I said, fuzzing works with oss-fuzz just fine without it.
>
> I'm inclined to take this as "it appears to work just fine". Fuzzed
> input register state may be lost by doing a library call somewhere,
> rendering the fuzzing results less useful. This would pretty
> certainly stop being tolerable the moment you compared results of
> native execution of a sequence of instructions with the emulated
> counterpart.

Yea, that may be. Any suggested way to fix the linking issue though?
I'm not even sure why the problem only appears in the oss-fuzz build,
when I just run make normally it seems to work.

Tamas