Re: [RFC PATCH v3 14/20] configure: use 'mkvenv ensure meson' to bootstrap meson

[John Snow]

On Mon, Apr 24, 2023 at 4:36 PM Warner Losh <imp@bsdimp.com> wrote:
>
>
>
> On Mon, Apr 24, 2023, 2:03 PM John Snow <jsnow@redhat.com> wrote:
>>
>> This commit changes how we detect and install meson. It notably removes
>> '--meson='.
>>
>> The previous patch creates a lightweight Python virtual environment
>> unconditionally using the user's configured $python that inherits system
>> packages. If Meson is installed there and meets our minimum version
>> requirements, we will use that Meson.
>>
>> In the event that Meson is installed but *not for the chosen Python
>> interpreter*, not found, or of insufficient version, we will attempt to
>> install Meson from vendored source into the newly created Python virtual
>> environment. This vendored installation is considered to replace the
>> mechanism from prior tarball distributions.
>>
>> This commit restores the ability to use a system meson, but in turn
>> temporarily removes the ability to use a meson as obtained from the
>> internet at configure-time (git submodules, as it stood prior to this
>> patch); that ability will be restored in the next commit.
>>
>> As a result of this patch, the Python interpreter we use for both our
>> own build scripts *and* Meson extensions are always known to be the
>> exact same Python. As a further benefit, there will also be a symlink
>> available in the build directory that points to the correct, configured
>> python and can be used by e.g. manual tests to invoke the correct,
>> configured Python unambiguously.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>>  configure                           | 72 ++++++++---------------------
>>  .gitlab-ci.d/buildtest-template.yml |  4 +-
>>  2 files changed, 21 insertions(+), 55 deletions(-)
>>
>> diff --git a/configure b/configure
>> index 462fe604d6..e9947369b2 100755
>> --- a/configure
>> +++ b/configure
>> @@ -731,8 +731,6 @@ for opt do
>>    ;;
>>    --skip-meson) skip_meson=yes
>>    ;;
>> -  --meson=*) meson="$optarg"
>> -  ;;
>>    --ninja=*) ninja="$optarg"
>>    ;;
>>    --smbd=*) smbd="$optarg"
>> @@ -1016,7 +1014,6 @@ Advanced options (experts only):
>>    --cross-prefix-ARCH=PREFIX cross compiler prefix when building ARCH guest 
>> test cases
>>    --make=MAKE              use specified make [$make]
>>    --python=PYTHON          use specified python [$python]
>> -  --meson=MESON            use specified meson [$meson]
>>    --ninja=NINJA            use specified ninja [$ninja]
>>    --smbd=SMBD              use specified smbd [$smbd]
>>    --with-git=GIT           use specified git [$git]
>> @@ -1089,7 +1086,8 @@ fi
>>
>>  # Resolve PATH
>>  python="$(command -v "$python")"
>> -explicit_python=yes
>> +# This variable is intended to be used only for error messages:
>> +target_python=$python
>>
>>  # Create a Python virtual environment using our configured python.
>>  # The stdout of this script will be the location of a symlink that
>> @@ -1101,7 +1099,6 @@ explicit_python=yes
>>  # - venv is cleared if it exists already;
>>  # - venv is allowed to use system packages;
>>  # - all setup is performed **offline**;
>> -# - No packages are installed by default;
>>  # - pip is not installed into the venv when possible,
>>  #   but ensurepip is called as a fallback when necessary.
>>
>> @@ -1116,58 +1113,27 @@ fi
>>  # Suppress writing compiled files
>>  python="$python -B"
>>
>> -has_meson() {
>> -  local python_dir=$(dirname "$python")
>> -  # PEP405: pyvenv.cfg is either adjacent to the Python executable
>> -  # or one directory above
>> -  if test -f $python_dir/pyvenv.cfg || test -f $python_dir/../pyvenv.cfg; 
>> then
>> -    # Ensure that Meson and Python come from the same virtual environment
>> -    test -x "$python_dir/meson" &&
>> -      test "$(command -v meson)" -ef "$python_dir/meson"
>> -  else
>> -    has meson
>> -  fi
>> -}
>>
>> -if test -z "$meson"; then
>> -    if test "$explicit_python" = no && has_meson && version_ge "$(meson 
>> --version)" 0.61.5; then
>> -        meson=meson
>> -    elif test "$git_submodules_action" != 'ignore' ; then
>> -        meson=git
>> -    elif test -e "${source_path}/meson/meson.py" ; then
>> -        meson=internal
>> -    else
>> -        if test "$explicit_python" = yes; then
>> -            error_exit "--python requires using QEMU's embedded Meson 
>> distribution, but it was not found."
>> -        else
>> -            error_exit "Meson not found.  Use --meson=/path/to/meson"
>> -        fi
>> +if ! $python "${source_path}/python/scripts/mkvenv.py" ensure \
>> +     --dir "${source_path}/python/wheels" \
>> +     "meson>=0.61.5" ;
>> +then
>> +    # We're very out of luck. Try to give a good diagnostic.
>> +    if test -e pyvenv/bin/meson; then
>> +        echo "Meson is too old:
>
>
> Does a minimum version still get printed? I've needed to know that in the 
> past when I got the error...
>
> Warner
>  $(pyvenv/bin/meson --version)"

At the end of the series, here's what happens if i change the meson
requirement to a fictionally too-high version that would be impossible
to fulfill:

--------------------------------------------------------------------------------
jsnow@scv ~/s/q/b/git (python-configure-venv)> ../../configure
--enable-debug --disable-pypi;
python determined to be '/usr/bin/python3'
python version: Python 3.11.2
MKVENV pyvenv
MKVENV ensure meson>=2.61.5
ERROR: Could not find a version that satisfies the requirement
meson>=2.61.5 (from versions: none)
ERROR: No matching distribution found for meson>=2.61.5

*** Ouch! ***

Could not ensure availability of 'meson>=2.61.5':
 • Python package 'meson' version '1.0.1' was found, but isn't suitable.
 • No suitable version found in, or failed to install from
'/home/jsnow/src/qemu/python/wheels'.
 • mkvenv was configured to operate offline and did not check PyPI.
--------------------------------------------------------------------------------