Re: [Bug-wget] fuzz tests

[Tim Rühsen]

Hi Nam,

not sure why, but your email just popped up here though it is dated
18.2.2019 9:39 AM.

On 2/18/19 9:39 AM, Nam Nguyen wrote:
> I am trying to version bump wget to 1.20.1. While installation works, I
> am trying to get `make check' to work on OpenBSD.
> 
> I need some help understanding the fuzz tests and their expected
> behavior. Are fuzzing tests supposed to try to crash the program with
> random inputs to uncover programming errors?

A 'make check' runs the fuzzers as regression tests with the test data
files found in each fuzzer's data directory (fuzz/<fuzzername>.in/* and
fuzz/<fuzzername>.repro/* for former crash reproducers).

We exclude the fuzzer test data from the tarball to not blow it up too
much. These test are mainly interesting for developers when using ASAN /
UBSAN builds (or valgrind test runs).

The test data comes mainly from OSS-Fuzz and covers all kinds of code
paths (though we are far away from 100%).

> I am getting a signal 6 (ENXIO?) and mostly signal 5 (EIO?).  Signal 6
> seems to be related to the stack smash protector feature of OpenBSD.
> All eight tests dump core files because they receive these signals.

Sounds like I have to start my OpenBSD VM (OpenBSD 5, not used since
years...) or install a new one :-)

> I attached `ports', `config.log' and `fuzz/test-suite.log'. `ports' is
> the log produced by the OpenBSD ports system when I run `make test'
> which should run all check targets. Note that `ports' reports a failure
> because it cannot find the fuzz tests, which are not included with the
> tarball. I had to clone the git repo and copy fuzz/*.in and fuzz/*.repro
> directories over before running `make check'.

'make check' should succeed when the fuzz data files are missing. I'll
check that. You could also reduce testing to 'make check -C tests' and
'make check -C testenv'.

> I am including some sample diffs that I needed to get `make test' to
> run.

Great, thank you. We normally don't build/test on *BSD, stopped with
OpenBSD 5 when general ports builds had many build issues and reporting
hit deaf ears. But I will try again and your work sounds like a great help !

> patch-fuzz_Makefile_am: -ldl doesn't exist on OpenBSD; libc handles it.
> patch-fuzz_wget_cookie_fuzzer_c: close stderr differently to avoid
> assigning to lvalue

The fuzzers have mainly been built for GNU/Linux / clang / OSS-Fuzz, so
there is some hard-coded stuff. We have to move that to configure.ac.

> patch-lib_Makefile_am: add unknown symbols to libgnu
> 
> Sorry for the long e-mail; I mainly want to understand the regression
> tests available for wget. Thank you.

Thank you for all the work :-)

The 'bt' output could be more infomative if wget was built with
debugging info (Line numbers etc).

With Best Regards, Tim

> 
> Best Regards,
> Nam
> 
> wget_css_fuzzer.c
> --8<---------------cut here---------------start------------->8---
>   exit status:134
>   Program terminated with signal 6, Aborted.
> 
>   $ doas -u _pbuild gdb fuzz/wget_css_fuzzer fuzz/wget_css*.core          
>   GNU gdb 6.3
> 
>   Core was generated by `wget_css_fuzzer'.
>   ...
>   #0  thrkill () at -:3
>   3       -: No such file or directory.
>           in -
>   (gdb) bt
>   #0  thrkill () at -:3
>   #1  0x00000a67fdad341c in __stack_smash_handler (func=Variable "func" is 
> not available.
>   )
>       at /usr/src/lib/libc/sys/stack_protector.c:79
>   #2  0x00000a65d1b8a49b in LLVMFuzzerTestOneInput ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_css_fuzzer
>   #3  0x00000a65d1b58ac0 in ?? ()
> --8<---------------cut here---------------end--------------->8---
> 
> wget_html_fuzzer.c
> --8<---------------cut here---------------start------------->8---
>   exit status: 133
>   Program terminated with signal 5, Trace/breakpoint trap.
> 
>   $ doas -u _pbuild gdb fuzz/wget_html_fuzzer fuzz/wget_html*.core 
>   GNU gdb 6.3
>   Core was generated by `wget_html_fuzzer'.
>   Program terminated with signal 5, Trace/breakpoint trap.
>   Reading symbols from /usr/lib/libpthread.so.26.1...done.
>   ...
>   #0  0x00000552f4f68375 in exit ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_html_fuzzer
>   (gdb) bt
>   #0  0x00000552f4f68375 in exit ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_html_fuzzer
>   #1  0x00000552f4f68133 in ___start ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_html_fuzzer
>   #2  0x0000000000000000 in ?? ()
> --8<---------------cut here---------------end--------------->8---
> 
> wget_cookie_fuzzer.c
> --8<---------------cut here---------------start------------->8---
> 
>   Trace/BPT trap
>   exit status: 133
>   Program terminated with signal 5, Trace/breakpoint trap
> 
>   $ doas -u _pbuild gdb fuzz/wget_cookie_fuzzer fuzz/wget_cookie*.core 
>   GNU gdb 6.3
>   ...
>   Core was generated by `wget_cookie_fuzz'.
>   Program terminated with signal 5, Trace/breakpoint trap.
>   Reading symbols from /usr/lib/libpthread.so.26.1...done.
>   ...
>   #0  0x00000c4a97be1385 in exit ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_cookie_fuzzer
>   (gdb) bt
>   #0  0x00000c4a97be1385 in exit ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_cookie_fuzzer
>   #1  0x00000c4a97be1133 in ___start ()
>      from 
> /mnt/playground/ports/pobj/wget-1.20.1/wget-1.20.1/fuzz/wget_cookie_fuzzer
>   #2  0x0000000000000000 in ?? ()
> --8<---------------cut here---------------end--------------->8---
> 
> patch-fuzz_Makefile_am
> --8<---------------cut here---------------start------------->8---
> $OpenBSD$
> 
> Index: fuzz/Makefile.am
> --- fuzz/Makefile.am.orig
> +++ fuzz/Makefile.am
> @@ -5,8 +5,7 @@ LDADD = ../lib/libgnu.a \
>   $(GETADDRINFO_LIB) $(HOSTENT_LIB) $(INET_NTOP_LIB) $(INET_PTON_LIB) \
>   $(LIBSOCKET) $(LIB_CLOCK_GETTIME) $(LIB_CRYPTO) $(LIB_GETLOGIN) 
> $(LIB_NANOSLEEP) $(LIB_POLL) \
>   $(LIB_POSIX_SPAWN) $(LIB_PTHREAD_SIGMASK) $(LIB_SELECT) $(LIBICONV) 
> $(LIBINTL) \
> - $(LIBMULTITHREAD) $(LIBTHREAD) $(SERVENT_LIB) @INTL_MACOSX_LIBS@ \
> - -ldl
> + $(LIBMULTITHREAD) $(LIBTHREAD) $(SERVENT_LIB) @INTL_MACOSX_LIBS@
>  
>  WGET_TESTS = \
>   wget_css_fuzzer$(EXEEXT) \
> --8<---------------cut here---------------end--------------->8---
> 
> patch-fuzz_wget_cookie_fuzzer_c
> --8<---------------cut here---------------start------------->8---
> $OpenBSD$
> 
> Index: fuzz/wget_cookie_fuzzer.c
> --- fuzz/wget_cookie_fuzzer.c.orig
> +++ fuzz/wget_cookie_fuzzer.c
> @@ -25,6 +25,8 @@
>  #include <stdio.h>  // fmemopen
>  #include <string.h>  // strncmp
>  #include <stdlib.h>  // free
> +#include <fcntl.h> // open
> +#include <unistd.h> // close, dup, dup2
>  
>  #include "wget.h"
>  #undef fopen_wgetrc
> @@ -68,7 +70,7 @@ void exit(int status)
>  
>  int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
>  {
> -     FILE *bak;
> +     int bak, fd;
>       struct cookie_jar *cookie_jar;
>       char *set_cookie;
>  
> @@ -79,8 +81,9 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t
>       memcpy(set_cookie, data, size);
>       set_cookie[size] = 0;
>  
> -     bak = stderr;
> -     stderr = fopen("/dev/null", "w");
> +     bak = dup(STDERR_FILENO);
> +     fd = open("/dev/null", O_WRONLY);
> +     dup2(fd, STDERR_FILENO);
>  
>       cookie_jar = cookie_jar_new();
>       cookie_handle_set_cookie(cookie_jar, "x", 81, "p", set_cookie);
> @@ -88,8 +91,8 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t
>       cookie_handle_set_cookie(cookie_jar, "x", 80, "p/d/", set_cookie);
>       cookie_jar_delete(cookie_jar);
>  
> -     fclose(stderr);
> -     stderr = bak;
> +     dup2(bak, STDERR_FILENO);
> +     close(bak);
>  
>          free(set_cookie);
>  --8<---------------cut here---------------end--------------->8---
> 
> patch-lib_Makefile_am
> --8<---------------cut here---------------start------------->8---
> $OpenBSD$
> 
> Index: lib/Makefile.am
> --- lib/Makefile.am.orig
> +++ lib/Makefile.am
> @@ -3114,17 +3114,13 @@ EXTRA_DIST += unicase/cased.h unicase/caseprop.h unict
>  
>  ## begin gnulib module unicase/empty-prefix-context
>  
> -if LIBUNISTRING_COMPILE_UNICASE_EMPTY_PREFIX_CONTEXT
>  libgnu_a_SOURCES += unicase/empty-prefix-context.c
> -endif
>  
>  ## end   gnulib module unicase/empty-prefix-context
>  
>  ## begin gnulib module unicase/empty-suffix-context
>  
> -if LIBUNISTRING_COMPILE_UNICASE_EMPTY_SUFFIX_CONTEXT
>  libgnu_a_SOURCES += unicase/empty-suffix-context.c
> -endif
>  
>  ## end   gnulib module unicase/empty-suffix-context
>  
> @@ -3447,9 +3443,7 @@ EXTRA_DIST += unistr.in.h
>  
>  ## begin gnulib module unistr/u8-cpy
>  
> -if LIBUNISTRING_COMPILE_UNISTR_U8_CPY
>  libgnu_a_SOURCES += unistr/u8-cpy.c
> -endif
>  
>  EXTRA_DIST += unistr/u-cpy.h
>  
> @@ -3457,9 +3451,7 @@ EXTRA_DIST += unistr/u-cpy.h
>  
>  ## begin gnulib module unistr/u8-mbtouc-unsafe
>  
> -if LIBUNISTRING_COMPILE_UNISTR_U8_MBTOUC_UNSAFE
>  libgnu_a_SOURCES += unistr/u8-mbtouc-unsafe.c unistr/u8-mbtouc-unsafe-aux.c
> -endif
>  
>  ## end   gnulib module unistr/u8-mbtouc-unsafe
>  
> @@ -3473,9 +3465,7 @@ endif
>  
>  ## begin gnulib module unistr/u8-uctomb
>  
> -if LIBUNISTRING_COMPILE_UNISTR_U8_UCTOMB
>  libgnu_a_SOURCES += unistr/u8-uctomb.c unistr/u8-uctomb-aux.c
> -endif
>  
>  ## end   gnulib module unistr/u8-uctomb
> --8<---------------cut here---------------end--------------->8---
>

signature.asc
Description: OpenPGP digital signature