Re: [PATCH for-5.1] Add GitHub action for Windows build

[Philippe Mathieu-Daudé]

On 8/3/20 10:20 PM, Stefan Weil wrote:
> The GitHub action is restricted to https://github.com/qemu/qemu.
> 
> Signed-off-by: Stefan Weil <sw@weilnetz.de>
> ---
> 
> This patch adds a GitHub action for continuous integration builds
> of QEMU for Windows.

Thanks for this!

> 
> The CI builds run on GitHub and include the WHPX code.
> The action rules avoid unnecessary waste of resources by
> limiting the GitHub action to the official mirror.
> 
> Peter, maybe this can still be added to 5.1.
> There should be no risk as it only adds the GitHub action.
> 
> Regards,
> Stefan
> 
> 
>  .github/workflows/build.sh                | 112 ++++++++++++++++++++++
>  .github/workflows/pkg-config-crosswrapper |  37 +++++++
>  .github/workflows/win.yml                 |  34 +++++++

This misses the MAINTAINERS entry:

-- >8 --
diff --git a/MAINTAINERS b/MAINTAINERS
index 0886eb3d2b..e64315265e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3068,6 +3068,12 @@ R: Wainer dos Santos Moschetta <wainersm@redhat.com>
 S: Maintained
 F: .gitlab-ci.yml

+GitHub actions (Windows installer)
+M: Stefan Weil <sw@weilnetz.de>
+M: Sunil Muthuswamy <sunilmut@microsoft.com>
+S: Maintained
+F: .github/workflows/
+
 Guest Test Compilation Support
 M: Alex Bennée <alex.bennee@linaro.org>
 R: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
>  3 files changed, 183 insertions(+)
>  create mode 100755 .github/workflows/build.sh
>  create mode 100755 .github/workflows/pkg-config-crosswrapper
>  create mode 100644 .github/workflows/win.yml
> 
> diff --git a/.github/workflows/build.sh b/.github/workflows/build.sh
> new file mode 100755
> index 0000000000..c430a367be
> --- /dev/null
> +++ b/.github/workflows/build.sh
> @@ -0,0 +1,112 @@
> +#!/bin/bash
> +
> +# GitHub actions - Create QEMU installer for Windows
> +
> +# Author: Stefan Weil (2020)
> +
> +#~ set -e
> +set -x
> +
> +ARCH=$1
> +DLLS="libgcc_s_sjlj-1.dll libgomp-1.dll libstdc++-6.dll"
> +
> +if test "$ARCH" != "i686"; then
> +  ARCH=x86_64
> +  DLLS="libgcc_s_seh-1.dll libgomp-1.dll libstdc++-6.dll"
> +fi
> +
> +ROOTDIR=$PWD
> +DISTDIR=$ROOTDIR/dist
> +HOST=$ARCH-w64-mingw32
> +BUILDDIR=bin/ndebug/$HOST
> +PKG_ARCH=mingw64-${ARCH/_/-}
> +
> +# Install cygwin key and add cygwin sources.
> +curl -s https://qemu.weilnetz.de/debian/gpg.key | sudo apt-key add -
> +echo deb https://qemu.weilnetz.de/debian/ testing contrib | \
> +  sudo tee /etc/apt/sources.list.d/cygwin.list
> +
> +# Install packages.
> +sudo apt-get update
> +sudo apt-get install --no-install-recommends \
> +  mingw-w64-tools nsis \
> +  gcc libc6-dev \
> +  g++-mingw-w64-${ARCH/_/-} gcc-mingw-w64-${ARCH/_/-} \
> +  bison flex gettext python3-sphinx texinfo \
> +  $PKG_ARCH-adwaita-icon-theme $PKG_ARCH-cogl $PKG_ARCH-curl \
> +  $PKG_ARCH-gmp $PKG_ARCH-gnutls $PKG_ARCH-gtk3 $PKG_ARCH-icu \
> +  $PKG_ARCH-libxml2 $PKG_ARCH-ncurses $PKG_ARCH-sdl2 $PKG_ARCH-usbredir
> +
> +# Workaround for buggy cross pkg-config.
> +sudo ln -sf $PWD/.github/workflows/pkg-config-crosswrapper \
> +  /usr/bin/$HOST-pkg-config
> +
> +# Get header files for WHPX API from Mingw-w64 git master.
> +(
> +sudo mkdir -p /usr/$HOST/sys-include
> +cd /usr/$HOST/sys-include
> +SF_URLBASE=https://sourceforge.net
> +URL=$SF_URLBASE/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include
> +sudo curl -s -o winhvemulation.h $URL/winhvemulation.h?format=raw
> +sudo curl -s -o winhvplatform.h $URL/winhvplatform.h?format=raw
> +sudo curl -s -o winhvplatformdefs.h $URL/winhvplatformdefs.h?format=raw
> +sudo ln -s winhvemulation.h WinHvEmulation.h
> +sudo ln -s winhvplatform.h WinHvPlatform.h
> +sudo ln -s winhvplatformdefs.h WinHvPlatformDefs.h
> +)
> +
> +DLL_PATH=$PWD/dll/$HOST
> +
> +mkdir -p $DISTDIR
> +mkdir -p $DLL_PATH
> +
> +for dll in $DLLS; do
> +  ln -sf /usr/lib/gcc/$HOST/*-win32/$dll $DLL_PATH
> +done
> +
> +DLLS="iconv.dll libatk-1.0-0.dll libbz2-1.dll"
> +DLLS="$DLLS libcairo-2.dll libcairo-gobject-2.dll libcurl-4.dll"
> +DLLS="$DLLS libeay32.dll libepoxy-0.dll libexpat-1.dll"
> +DLLS="$DLLS libffi-6.dll libfontconfig-1.dll libfreetype-6.dll"
> +DLLS="$DLLS libgdk-3-0.dll libgdk_pixbuf-2.0-0.dll"
> +DLLS="$DLLS libgio-2.0-0.dll libglib-2.0-0.dll libgmodule-2.0-0.dll"
> +DLLS="$DLLS libgmp-10.dll libgnutls-30.dll libgobject-2.0-0.dll 
> libgtk-3-0.dll"
> +DLLS="$DLLS libharfbuzz-0.dll libhogweed-4.dll libidn2-0.dll libintl-8.dll"
> +DLLS="$DLLS libjpeg-8.dll liblzo2-2.dll"
> +DLLS="$DLLS libncursesw6.dll libnettle-6.dll libnghttp2-14.dll"
> +DLLS="$DLLS libp11-kit-0.dll libpango-1.0-0.dll libpangocairo-1.0-0.dll"
> +DLLS="$DLLS libpangoft2-1.0-0.dll libpangowin32-1.0-0.dll libpcre-1.dll"
> +DLLS="$DLLS libpixman-1-0.dll libpng16-16.dll libssh2-1.dll libtasn1-6.dll"
> +DLLS="$DLLS libunistring-2.dll libusb-1.0.dll libusbredirparser-1.dll"
> +DLLS="$DLLS SDL2.dll ssleay32.dll zlib1.dll"
> +
> +for dll in $DLLS; do
> +  ln -sf /usr/$HOST/sys-root/mingw/bin/$dll $DLL_PATH
> +done
> +
> +ln -sf /usr/$HOST/lib/libwinpthread-1.dll $DLL_PATH
> +
> +# Build QEMU installer.
> +
> +echo Building $HOST...
> +mingw=/usr/$HOST/sys-root/mingw
> +mkdir -p $BUILDDIR && cd $BUILDDIR
> +
> +# Run configure.
> +../../../configure --cross-prefix=$HOST- --disable-guest-agent-msi \
> +    --disable-werror --enable-whpx \
> +    --extra-cflags="-I $mingw/include" \
> +    --extra-ldflags="-L $mingw/lib"
> +
> +# Add config.log to build artifacts.
> +cp config.log $DISTDIR/
> +
> +make
> +
> +echo Building installers...
> +date=$(date +%Y%m%d)
> +INSTALLER=$DISTDIR/qemu-$ARCH-setup-$date.exe
> +make installer DLL_PATH=$DLL_PATH SIGNCODE=true INSTALLER=$INSTALLER
> +
> +echo Calculate SHA-512 checksum...
> +(cd $DISTDIR; exe=$(basename $INSTALLER); sha512sum $exe >${exe/exe/sha512})
> diff --git a/.github/workflows/pkg-config-crosswrapper 
> b/.github/workflows/pkg-config-crosswrapper
> new file mode 100755
> index 0000000000..768e554194
> --- /dev/null
> +++ b/.github/workflows/pkg-config-crosswrapper
> @@ -0,0 +1,37 @@
> +#! /bin/sh
> +# pkg-config wrapper for cross-building
> +# Sets pkg-config search path to search multiarch and historical 
> cross-compiling paths.
> +
> +# If the user has already set PKG_CONFIG_LIBDIR, believe it (even if empty):
> +# it's documented to be an override
> +if [ x"${PKG_CONFIG_LIBDIR+set}" = x ]; then
> +  # GNU triplet for the compiler, e.g. i486-linux-gnu for Debian i386,
> +  # i686-linux-gnu for Ubuntu i386
> +  basename="${0##*/}"
> +  triplet="${basename%-pkg-config}"
> +  # Normalized multiarch path if any, e.g. i386-linux-gnu for i386
> +  multiarch="`dpkg-architecture -t"${triplet}" -qDEB_HOST_MULTIARCH 
> 2>/dev/null`"
> +
> +  PKG_CONFIG_LIBDIR="/usr/local/${triplet}/lib/pkgconfig"
> +  # For a native build we would also want to append /usr/local/lib/pkgconfig
> +  # at this point; but this is a cross-building script, so don't
> +  PKG_CONFIG_LIBDIR="$PKG_CONFIG_LIBDIR:/usr/local/share/pkgconfig"
> +
> +  if [ -n "$multiarch" ]; then
> +    
> PKG_CONFIG_LIBDIR="/usr/local/lib/${multiarch}/pkgconfig:$PKG_CONFIG_LIBDIR"
> +    PKG_CONFIG_LIBDIR="$PKG_CONFIG_LIBDIR:/usr/lib/${multiarch}/pkgconfig"
> +  fi
> +
> +  PKG_CONFIG_LIBDIR="$PKG_CONFIG_LIBDIR:/usr/${triplet}/lib/pkgconfig"
> +  # For a native build we would also want to append /usr/lib/pkgconfig
> +  # at this point; but this is a cross-building script, so don't
> +  # If you want to allow use of un-multiarched -dev packages for crossing
> +  # (at the risk of finding build-arch stuff you didn't want, if not in a 
> clean chroot)
> +  # Uncomment the next line:
> +  # PKG_CONFIG_LIBDIR="$PKG_CONFIG_LIBDIR:/usr/lib/pkgconfig"
> +  PKG_CONFIG_LIBDIR="$PKG_CONFIG_LIBDIR:/usr/share/pkgconfig"
> +
> +  export PKG_CONFIG_LIBDIR
> +fi
> +
> +exec pkg-config "$@"
> diff --git a/.github/workflows/win.yml b/.github/workflows/win.yml
> new file mode 100644
> index 0000000000..81cf48530f
> --- /dev/null
> +++ b/.github/workflows/win.yml
> @@ -0,0 +1,34 @@
> +# GitHub actions - Create QEMU installers for Windows
> +
> +# The action is restricted to https://github.com/qemu/qemu.
> +# That avoids an unnecessary waste of resources when each fork
> +# runs the action, too.
> +
> +name: Cross build for Windows
> +
> +on: [push]
> +
> +jobs:
> +  build32:
> +    if: github.repository == 'qemu/qemu'
> +    runs-on: [ubuntu-20.04]

Since it is based on Ubuntu, we should be able to run it on GitLab
directly (and simplifies our CI). Also we could drop the Shippable
docker images too.

This works, so:
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

But I'm not keen on not using GitLab instead.

The only point I see of using GitHub/Azureus is if we then install
and run testing in the Windows Server 2019 environment:

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idruns-on

> +    steps:
> +    - uses: actions/checkout@v2
> +    - name: Build QEMU installer (32 bit)
> +      run: .github/workflows/build.sh i686
> +    - uses: actions/upload-artifact@v1
> +      with:
> +        name: QEMU Installer Windows 32 bit
> +        path: dist
> +
> +  build64:
> +    if: github.repository == 'qemu/qemu'
> +    runs-on: [ubuntu-20.04]
> +    steps:
> +    - uses: actions/checkout@v2
> +    - name: Build QEMU installer (64 bit)
> +      run: .github/workflows/build.sh x86_64
> +    - uses: actions/upload-artifact@v1
> +      with:
> +        name: QEMU Installer Windows 64 bit
> +        path: dist
>