Re: Debian archive rebuild with Autoconf 2.69c

[Zack Weinberg]

On Sat, Sep 26, 2020 at 2:32 AM Tomasz Kłoczko <kloczko.tomasz@gmail.com> wrote:
> On Thu, 24 Sep 2020 at 21:42, Zack Weinberg <zackw@panix.com> wrote:
>
> Today I've decided to give the chance autoconf 2.69c in my build env.

Thanks for trying it!

> The First package on which build failed was sssd.

Flood of obsolete-macro warnings aside, it looks like this is the
actual regression:

> checking for python2... no
> checking for python3... yes
> checking for python3... /usr/bin/python3
> checking whether /usr/bin/python3 version is >= 3.3... yes
> checking for /usr/bin/python3 version... 3.8
> checking for /usr/bin/python3 platform... linux
> checking for /usr/bin/python3 script directory... 
> ${prefix}/lib/python3.8/site-packages
> checking for /usr/bin/python3 extension module directory... 
> ${exec_prefix}/lib64/python3.8/site-packages
> checking for python3.8-config... /usr/bin/python3.8-config
> checking for headers required to compile python extensions... not found
> configure: error: Could not find python3 headers

versus

> ...
> checking for python3.8-config... /usr/bin/python3.8-config
> checking for headers required to compile python extensions... found

I can reproduce this problem on my computer.  It's one of the
potential backward incompatibilities warned about in NEWS:

   - Autoconf macros that use AC_REQUIRE internally, are not safe to
     use inside of hand-written shell conditional or looping
     constructs.  Use AS_IF, AS_CASE, AS_FOR, etc. instead.
     (See the “Prerequisite Macros” section of the manual for
     further explanation.)

Specifically, the configure script has

if test x$HAVE_PYTHON2_BINDINGS = xyes; then
   ...
   AM_CHECK_PYTHON_HEADERS([], [AC_MSG_ERROR([Could not find python2 headers])])
   ...
fi

if test x$HAVE_PYTHON3_BINDINGS = xyes; then
   ...
   AM_CHECK_PYTHON_HEADERS([], [AC_MSG_ERROR([Could not find python2 headers])])
   ...
fi

AM_CHECK_PYTHON_HEADERS uses AC_TRY_CPP, and that's the first thing in
the file that needs $CPP to be set, which is done by AC_PROG_CPP,
which AC_TRY_CPP invokes via AC_REQUIRE.

With autoconf 2.69, AC_PROG_CPP was expanded as a side effect of
AM_PROG_CC_C_O, so $CPP was set very early and this wasn't an issue.
But with 2.70, AC_PROG_CPP is only expanded the first time
AM_CHECK_PYTHON_HEADERS appears ... which is inside a shell `if` that
is invisible to M4.  So the actual code of AC_PROG_CPP is only run
when $HAVE_PYTHON2_BINDINGS is 'yes'.  But it _needs_ to be run when
_either_ $HAVE_PYTHON2_BINDINGS _or_ $HAVE_PYTHON3_BINDINGS is 'yes',
and it won't be emitted twice.

(This isn't because of AC_TRY_CPP being obsolete; you'd have the same
problem with AC_PREPROC_IFELSE.)

Changing both of these `if`s to `AS_IF`s fixes the problem, because
then the shell conditionals are visible to M4 and the expansion of
AC_PROG_CPP is placed outside (above) the first one.  The configure
script runs to completion on my machine with this patch applied:

--- configure.ac.269    2020-09-26 11:01:41.555520661 -0400
+++ configure.ac    2020-09-26 11:02:49.827375098 -0400
@@ -323,8 +323,8 @@
 AS_IF([test x$HAVE_PYTHON3 = xyes],
       [AC_PATH_PROG(PYTHON3, python3)])

-if test x$HAVE_PYTHON2_BINDINGS = xyes; then
-    AS_IF([test x$HAVE_PYTHON2 != xyes],
+AS_IF([test x$HAVE_PYTHON2_BINDINGS = xyes],
+   [AS_IF([test x$HAVE_PYTHON2 != xyes],
           [AC_MSG_ERROR([
 The program python2 was not found in search path.
 Please ensure that it is installed and its directory is included in the search
@@ -347,10 +347,10 @@
     AC_SUBST([PYTHON2_EXEC_PREFIX], [$PYTHON_EXEC_PREFIX])

     SSS_CLEAN_PYTHON_VARIABLES
-fi
+])

-if test x$HAVE_PYTHON3_BINDINGS = xyes; then
-    AS_IF([test x$HAVE_PYTHON3 != xyes],
+AS_IF([test x$HAVE_PYTHON3_BINDINGS = xyes],
+   [AS_IF([test x$HAVE_PYTHON3 != xyes],
           [AC_MSG_ERROR([
 The program python3 was not found in search path.
 Please ensure that it is installed and its directory is included in the search
@@ -373,7 +373,7 @@
     AC_SUBST([PYTHON3_EXEC_PREFIX], [$PYTHON_EXEC_PREFIX])

     SSS_CLEAN_PYTHON_VARIABLES
-fi
+])

 if test x$HAVE_PYTHON3 = xyes; then
     PYTHON_EXEC=$PYTHON3

However, I see a whole lot of other M4 macros in this script being
invoked inside of hand-written shell `if`s.  I'm going to file a bug
on SSSD.

We could address this particular failure in Autoconf by having the
*shell function* `as_fn_c_try_cpp` run the code of AC_PROG_CPP if it
notices that $CPP is empty.  However, not all of the potential
problems with AC_REQUIRE inside shell conditionals can be addressed
this way.  Also, running AC_PROG_CPP from inside as_fn_try_cpp is not
necessarily safe.  For instance, in this code AC_TRY_CPP is invoked
with CPPFLAGS temporarily overridden.  We would not want a temporary
setting of CPPFLAGS to get captured by AC_PROG_CPP and written into
config.status.  On the whole I do not think that change would be a
good idea.

Does anyone have a better idea?

zw