Re: [PATCH] bootstrap: When a commit hash is specified, do a shallow fetch if possible

[Daniel Kiper]

On Thu, Oct 21, 2021 at 12:49:19PM -0500, Glenn Washburn wrote:
> The gnulib sources are large but more importantly have lots of changes. So
> initial checkout of the repository can take a long time when network or
> cpu resources are limited. The later is especially acute in a non-KVM QEMU
> virtual machine (which can take 40+ minutes compared to <30 seconds with
> this change[1]). The problem is specific to how GRUB uses gnulib, which is
> by pegging the desired checkout to a specific git revision. In this case,
> git can not do a shallow clone by using the --depth option because git does
> not know ahead of time how deep the revision is from the tip. So git must
> clone the whole repository.
>
> However, there is an alternate method that requires support from the git
> server[2], namely by asking for a specific commit on fetch. Refactor to use
> fetch and fallback to fetching the entire repository if fetching by commit
> hash fails.
>
> Currently the git server hosting the official gnulib git repository does not
> support fetch by commit hash[3]. However, there are mirrors which do support
> this[4], and can be specified by setting the $GNULIB_URL.
>
> [1] https://savannah.nongnu.org/support/index.php?110553#comment1
> [2] https://stackoverflow.com/a/3489576/2108011
> [3] https://savannah.nongnu.org/support/index.php?110553
> [4] https://github.com/coreutils/gnulib
>
> Signed-off-by: Glenn Washburn <development@efficientek.com>
> ---
>  bootstrap | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/bootstrap b/bootstrap
> index 5b08e7e2d..914f911f8 100755
> --- a/bootstrap
> +++ b/bootstrap
> @@ -665,9 +665,21 @@ if $use_gnulib; then
>        shallow=
>        if test -z "$GNULIB_REVISION"; then
>          git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 
> 2'
> +        git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" 
> \
> +          || cleanup_gnulib
> +      else
> +        git fetch -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 
> 2'
> +        mkdir -p "$gnulib_path"
> +        git -C "$gnulib_path" init
> +        git -C "$gnulib_path" remote add origin 
> ${GNULIB_URL:-$default_gnulib_url}
> +        # Can not do a shallow fetch if fetch by commit hash fails because we
> +        # do not know how the to go to get to $GNULIB_REVISION, so we must 
> get
> +        # all commits.
> +        git -C "$gnulib_path" fetch $shallow origin "$GNULIB_REVISION" \
> +          || git -C "$gnulib_path" fetch origin \
> +          || cleanup_gnulib
> +        git -C "$gnulib_path" reset --hard FETCH_HEAD

Is this hunk from gnulib upstream? If not I would prefer if you upstream
it into gnulib. We should avoid custom patches for imported code as much
as possible.

Daniel