[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] gitlab-ci.yml: Add oss-fuzz build tests
From: |
Thomas Huth |
Subject: |
Re: [PATCH] gitlab-ci.yml: Add oss-fuzz build tests |
Date: |
Fri, 17 Jul 2020 07:40:31 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 |
On 16/07/2020 18.33, Alexander Bulekov wrote:
> This tries to build and run the fuzzers with the same build-script used
> by oss-fuzz. This doesn't guarantee that the builds on oss-fuzz will
> also succeed, since oss-fuzz provides its own compiler and fuzzer vars,
> but it can catch changes that are not compatible with the the
> ./scripts/oss-fuzz/build.sh script.
> The strange way of finding fuzzer binaries stems from the method used by
> oss-fuzz:
> https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-runner/targets_list
>
> Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
> ---
>
> Similar to Thomas' patch:
>
>> Note: This patch needs two other patches merged first to work correctly:
>
>> - 'fuzz: Expect the cmdline in a freeable GString' from Alexander
>
>> - 'qom: Plug memory leak in "info qom-tree"' from Markus
>
> Otherwise the test will fail due to detected memory leaks.
>
> Fair warning: I haven't been able to trigger this new job yet. I tried
> to run the pipeline with these changes on my forked repo on gitlab, but
> did not reach the build-oss-fuzz. I think this is due to some failures
> in the Containers-layer-2 stage:
>
> ...
> Error response from daemon: manifest for
> registry.gitlab.com/a1xndr/qemu/qemu/debian-all-test-cross:latest not
> found: manifest unknown: manifest unknown
> #2 [internal] load .dockerignore
> #2 transferring context:
> #2 transferring context: 2B 0.1s done
> #2 DONE 0.1s
> #1 [internal] load build definition from tmpg8j4xoop.docker
> #1 transferring dockerfile: 2.21kB 0.1s done
> #1 DONE 0.2s
> #3 [internal] load metadata for docker.io/qemu/debian10:latest
> #3 ERROR: pull access denied, repository does not exist or may require
> authorization: server message: insufficient_scope: authorization failed
These look like the problems that we've seen with the main repo until
two days ago, too, e.g.:
https://gitlab.com/qemu-project/qemu/-/jobs/640410842
Maybe Alex (Bennée) can comment on how to resolve them?
>
> .gitlab-ci.yml | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index e96f8794b9..a50df420c9 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -182,6 +182,20 @@ build-fuzzer:
> || exit 1 ;
> done
As mentioned in my other mail, I think you can replace my build-fuzzer
job once this is working.
> +build-oss-fuzz:
> + <<: *native_build_job_definition
> + variables:
> + IMAGE: fedora
> + script:
> + - OUT_DIR="./build" CC=clang-9 CXX=clang++-9 CFLAGS="-fsanitize=address"
> + LIB_FUZZING_ENGINE="-fsanitize=fuzzer" CFL
That "CFL" at the end seems to be a typo (leftover from "CFLAGS")?
Also the fedora container does not have clang-9 :
https://gitlab.com/huth/qemu/-/jobs/643383032#L28
I think it is at clang 10 already, so maybe just use CC=clang (without
version number)?
> + ./scripts/oss-fuzz/build.sh
> + - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f);
> do
> + grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue
> ;
> + echo Testing ${fuzzer} ... ;
> + "${fuzzer}" -runs=1000 || exit 1 ;
> + done
Should we exclude the virtio-net tests, since they could leak network
traffic to the host?
Thomas