Re: [Qemu-devel] [PATCH] script: git script to compile every commit in a range of commits

[Peter Crosthwaite]

Hi Jeff,

On Sat, Jun 1, 2013 at 2:39 AM, Jeff Cody <address@hidden> wrote:
> This is a git script that will iterate through every commit in a
> specified range, and perform a configure and make.  The intention of
> this script is not to act as a check of code correctness, but to see if
> any commit breaks compilation of the tree.
>

Should we possibly throw a checkpatch in there?

> The idea is that prior to submitting a patch or patch series, the
> submitter should verify that no patch in the series breaks the build,
> thereby breaking git-bisect.  This script may also be useful for
> reviewers/maintainers dealing with multi-patch series.
>
> The range passed in to the script is in the form of a standard git range;
> the default is HEAD^!  (i.e. just the latest commit).
>
> Options may be passed in for configure, as well as make.  The log output
> filename and directory may also be optionally specified.  All options
> are stored via git-config.
>
> If everything compiled without error, then the exit code from the
> program will be 0.  Upon error, the stats for the offending
> commit will be shown.
>
> Signed-off-by: Jeff Cody <address@hidden>
> ---
>  scripts/git-compile-check | 202 
> ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 202 insertions(+)
>  create mode 100755 scripts/git-compile-check
>
> diff --git a/scripts/git-compile-check b/scripts/git-compile-check
> new file mode 100755
> index 0000000..4b90f91
> --- /dev/null
> +++ b/scripts/git-compile-check
> @@ -0,0 +1,202 @@
> +#!/bin/bash
> +#
> +# See $desc, below, for program description
> +#
> +# Copyright (c) 2013 Red Hat, Inc.
> +#
> +# Author(s):
> +#   Jeff Cody <address@hidden>
> +#
> +# This program is free software; you can redistribute it and/or modify it 
> under
> +# the terms of the GNU General Public License as published by the Free 
> Software
> +# Foundation; under version 2 of the license
> +#
> +# This program is distributed in the hope that it will be useful, but WITHOUT
> +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 
> FITNESS
> +# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
> +# details.
> +#
> +# You should have received a copy of the GNU General Public License along 
> with
> +# this program; if not, see <http://www.gnu.org/licenses/gpl-2.0.html>.
> +#
> +
> +set -C -u -e
> +set -o pipefail
> +
> +desc="
> +$0 iterates through a git commit range, and performs the
> +following on each commit:
> +  - git checkout
> +  - make clean
> +  - configure
> +  - make
> +
> +It will also optionally perform a git-reset and git-clean between
> +checkouts, if requested via the '-f' option.
> +
> +The script will exit and report on first error on any of the above steps,
> +(except no error checking is performed on 'make clean')
> +
> +NOTE: While executing, the script will checkout out each commit
> +      in the range in the current git tree.  On exit, the HEAD
> +      at the time the script was called is checked out"
> +
> +
> +# default range is the last commit
> +def_range="HEAD^!"
> +def_config_opt="--target-list=x86_64-softmmu"
> +# you may want to have make perform multiple jobs, e.g. -j4
> +# this is ommitted as the default in case the project makefile
> +# is not safe for parallel make processes
> +def_make_opt=""
> +def_log="output-$$.log"
> +def_logdir=""
> +force_clean='n'
> +
> +logfile=$def_log
> +range=`git config compile-check.range || true`
> +config_opt=`git config compile-check.configopt || true`
> +make_opt=`git config compile-check.makeopt || true`
> +logdir=`git config compile-check.logdir || true`
> +
> +if [[ -z "$range" ]]
> +then
> +    range=$def_range
> +    git config compile-check.range $range || true
> +fi
> +if [[ -z "$config_opt" ]]
> +then
> +    config_opt=$def_config_opt
> +    git config compile-check.configopt $config_opt || true
> +fi
> +if [[ -z "$make_opt" ]]
> +then
> +    make_opt=$def_make_opt
> +    git config compile-check.makeopt $make_opt || true
> +fi
> +if [[ -z "$logdir" ]]
> +then
> +    logdir=$def_logdir
> +    git config compile-check.logdir $logdir || true
> +fi
> +
> +usage() {
> +    echo ""
> +    echo "$0 [OPTIONS]"
> +    echo "$desc"
> +    echo ""
> +    echo "OPTIONS:"
> +    echo "      -r git range
> +            optional; default is '$range'
> +                    "
> +    echo "      -c configure options
> +            optional; default is '$config_opt'
> +                    "
> +    echo "      -m make options
> +            optional; default is '$make_opt'
> +                    "
> +    echo "      -d log dir
> +            optional; default is '$logdir'
> +                    "
> +    echo "      -l log filename
> +            optional; default is output-PROCID, where PROCID is the bash 
> process id
> +            note: you may specify a full path for the log filename here, and 
> exclude the
> +            -d option.
> +                    "
> +    echo "      -f force a git reset and clean
> +            this will cause a 'git reset --hard; git clean -fdx' to be run 
> between checkouts.
> +            !! WARNING: This may cause data loss in your git tree.
> +                        READ THE git-clean and git-reset man pages and make
> +                        sure you understand the implications of
> +                        'git clean -fdx' and 'git reset --hard' before using 
> !!
> +                    "
> +    echo "      -h help"
> +}
> +
> +while getopts ":r:c:m:l:d:hf" opt
> +do
> +    case $opt in
> +        r) range=$OPTARG
> +            ;;
> +        c) config_opt=$OPTARG
> +            ;;
> +        m) make_opt=$OPTARG
> +            ;;
> +        d) logdir=$OPTARG
> +            ;;
> +        l) logfile=$OPTARG
> +            ;;
> +        f) force_clean='y'
> +            ;;
> +        h) usage
> +           exit
> +            ;;
> +        \?) echo "Unknown option: -$OPTARG" >&2
> +            usage
> +            exit 1
> +            ;;
> +    esac
> +done
> +
> +# append a '/' to logdir if $logdir was specified without one
> +[[ -n "$logdir" ]] && [[ ${logdir:${#logdir}-1} != "/" ]] && 
> logdir="${logdir}/"
> +
> +logfile="${logdir}${logfile}"
> +
> +head=`git rev-parse HEAD`
> +total=`git rev-list "$range" |wc -l`
> +
> +echo "log output: $logfile"
> +
> +rm -f "$logfile"
> +date > "$logfile"
> +echo "git compile check for $range." >> "$logfile"
> +echo "* configure options='$config_opt'" >> "$logfile"
> +echo "* make options='$make_opt'" >> "$logfile"
> +echo "Performing a test compile on $total patches" | tee -a "$logfile"
> +echo "-------------------------------------------------------------" >> 
> "$logfile"
> +echo "" | tee -a "$logfile"
> +
> +clean_repo() {
> +    if [[ $force_clean == 'y' ]]
> +    then
> +        git reset --hard >> "$logfile" 2>&1 || true
> +        git clean -fdx -e "$logfile" >> "$logfile" 2>&1 || true
> +    fi
> +}
> +
> +# we want to cleanup and return the git tree back to the previous head
> +trap cleanup EXIT
> +
> +cleanup() {
> +    echo ""
> +    echo -n "Cleaning up..."
> +    clean_repo
> +    git checkout $head > /dev/null 2>&1
> +    echo "done."
> +}
> +
> +cnt=1
> +# don't pipe the git job into read, to avoid subshells
> +while read hash
> +do
> +    txt=`git log --pretty=tformat:"%h: %s" $hash^!`
> +    echo "${cnt}/${total}: compiling: $txt" | tee -a "$logfile"
> +    let cnt=$cnt+1;
> +    echo "####################" >> "$logfile"
> +    clean_repo
> +    make clean > /dev/null 2>&1 || true

Can we opt out of the clean and configure on every patch? Most series
you know you can trust incremental builds for compile testing. Really
only need to be this defensive if playing with make or configure.

Regards,
Peter

> +    git checkout $hash          >> "$logfile" 2>&1 && \
> +        ./configure $config_opt >> "$logfile" 2>&1 && \
> +        make $make_opt          >> "$logfile" 2>&1 ||
> +    (
> +        echo "" | tee -a "$logfile"
> +        echo "ERROR: commit $hash failed to build!" | tee -a "$logfile"
> +        git show --stat $hash | tee -a "$logfile"
> +        exit 1
> +    )
> +done < <(git log $range --pretty=tformat:"%H" --reverse)
> +
> +echo "
> +All patches in $range compiled successfully!" | tee -a "$logfile"
> +exit 0
> --
> 1.8.1.4
>
>