Re: tweak for documentation of how to create universal binaries

[Ralf Wildenhues]

[ adding automake-patches; this is
  <http://thread.gmane.org/gmane.comp.sysutils.autoconf.bugs/6642> ]

Hello Bruno, Peter,

* Bruno Haible wrote on Sat, Apr 11, 2009 at 12:29:10PM CEST:
> Ralf Wildenhues wrote:
> > AIUI, the nature of the issue seems to be that one process scribbles
> > into the file already written to by another process.
> 
> Yes. Or that two processes (or two threads) write to the same file at the
> same time, through two different file descriptors.

Which means, there is a chance that the result may be a more complicated
mix of the outputs of both processes (say, when output is longer than a
couple of blocks).  Thus, this:

> > then we may still be able to exploit the fact that the output by the
> > second process ends in a newline without backslash.
> 
> Yes, this should be possible.

shouldn't be possible in general.

> Other alternatives would be to run the gcc command twice, once with a
> single -arch option only to produce the .TPo file, and once with all
> -arch options and without -MD, to produce the object file. But doubling
> the compilation times will not sit with most people. Then people will
> really prefer --disable-dependency-tracking.

This leads to another idea ... see below.

> Or to repeat the gcc command if and only if the generated .TPo file is
> malformed. The performance overhead of doing this should be small.

This could also be impossible to detect in general.

> > +      -e 's/^['$alpha']:\/[^:]*: / /' \
> > +      -e '$!s/[^\\]$/&\\/' '< "$tmpdepfile" >> "$depfile"
> 
> The last line has an odd number of single-quotes.

and my untested idea was broken too.  Yep.  Thanks for testing it.

* Peter O'Gorman wrote on Sat, Apr 11, 2009 at 07:13:22PM CEST:
> Ralf Wildenhues wrote:
> 
> > Well, if Bruno has run the latter command only (without going through
> > the depcomp wrapper), and got inconsistent results, then that indicates
> > a bug in this gcc.  Peter, would you file a bug report with Apple for us?
> 
> Well, if you put the multiple -arch flags in CFLAGS/CXXFLAGS instead of
> CC/CXX then configure will set depmode to gcc3 rather than gcc, and the
> first compile will error out with:
> 
> gcc-4.0: -E, -S, -save-temps and -M options are not allowed with
> multiple -arch flags

Ah ha!  Thank you for providing this bit of information!
It tells us two things:
- gcc3 depmode was rightly not chosen due to this erroring out,
- it is probably a bug that the _AM_DEPENDENCIES macro does not take
  $CFLAGS into account when testing for the dependency mode.

> When depmode is set to gcc, it looks like that check is avoided by
> quoting -MD with -Wp, - so I'm pretty sure that if I file a bug, it will
> be rejected.

I guess.  I don't want to remove the -Wp, though, it was certainly added
for a good reason, back in 1998 in commit 5de4fd8497418 in Automake.

> Dependency output is quite likely to be different for different
> architectures anyway...
> 
> If it were up to me, I'd just document the restriction as Bruno suggests.

I'm still not quite ready to give up.

AFAICS depcomp cannot detect breakage reliably, as it is bound to a race
condition.  However, depend.m4 could detect universal builds, and in
that case discard any problematic dependency extractors.  That means, in
effect that only "slow" extraction mechanisms are eligible (and will
thus be chosen only if --enable-dependency-tracking was passed
explicitly).

The patch below (against Automake) should do the trick.
Can you try it out, and report back the depmode chosen with and without
--enable-dependency-tracking and whether the depmode works (in case of
doubt, you can run depcomp7.test from the Automake testsuite)?
Can you also try to see if the "icc" depmode bit can be omitted from the
patch?

Thanks.

A next step could be to improve upon the situation, by doing one of the
following:

- drop all but the first -arch commands from the command used in the
dashmstdout depmode,

- add a new depmode right after gcc3 which uses something like what
Peter suggests:

> For recent versions of Apple's gcc, you can pass -Xarch_<arch> to pass
> flags to one architecture compile. So configure could check for multiple
> -arch flags, choose one, and quote every dependency flag etc with
> -Xarch_<arch>. e.g. -Xarch_i386 -MT -Xarch_i386 file.o -Xarch_i386  -MD
> -Xarch_i386 -MP -Xarch_i386 -MF -Xarch_i386 .deps/file.Tpo

so that at least newer gcc's can benefit.

- Take into account $CPPFLAGS $CFLAGS/CXXFLAGS $LDFLAGS when computing
the depmode.  I am a bit wary to take this step, as it is more likely
to regress other cases, and the first patch below isn't.

> I doubt that the effort is worth it.

We can improve step by step.

Cheers,
Ralf

    Avoid racy depmodes with universal builds.
    
    * m4/depend.m4 (_AM_DEPENDENCIES): If universal builds are used,
    avoid racy depmodes.
    Report by Bruno Haible, analysis by Bruno Haible, Peter O'Gorman,
    and Eric Blake.

diff --git a/m4/depend.m4 b/m4/depend.m4
index 451312a..3bc6952 100644
--- a/m4/depend.m4
+++ b/m4/depend.m4
@@ -63,6 +63,16 @@ AC_CACHE_CHECK([dependency style of $depcc],
   if test "$am_compiler_list" = ""; then
      am_compiler_list=`sed -n ['s/^#*\([a-zA-Z0-9]*\))$/\1/p'] < ./depcomp`
   fi
+  am__universal=false
+  m4_case($1, [CC],
+    [case " $depcc " in #(
+     *\ -arch\ *\ -arch\ *) am__universal=true ;;
+     esac],
+    [CXX],
+    [case " $depcc " in #(
+     *\ -arch\ *\ -arch\ *) am__universal=true ;;
+     esac])
+
   for depmode in $am_compiler_list; do
     # Setup a source with many dependencies, because some compilers
     # like to wrap large dependency lists on column 80 (with \), and
@@ -87,6 +97,10 @@ AC_CACHE_CHECK([dependency style of $depcc],
     am__obj=sub/conftest.${OBJEXT-o}
     am__minus_obj="-o $am__obj"
     case $depmode in
+    gcc | icc)
+      # These depmodes cause a compiler race in universal mode.
+      test "$am__universal" = false || continue
+      ;;
     nosideeffect)
       # after this tag, mechanisms are not by side-effect, so they'll
       # only be used when explicitly requested