Make run in parallel mode with output redirected to a regular file can randomly drop output lines

[Stefano Lattarini]

The issue is present both in all of make 3.81, make 3.82 and make
built from latest Git.  Here is a script that demonstrates it:

  $ cat foo.bash
  #!/bin/bash

  cat > GNUmakefile <<'END'
  all := 0 1 2 3 4 5 6 7 8 9
  default: $(all)
  $(all):
        @sleep 0.$$(($RANDOM % 10)); echo $@
  END

  : > exp
  for i in {0..9}; do echo $i >>exp; done

  lost_count=0
  for ((i = 0; i <= 150; i++)); do
    ${MAKE-make} -j > .t
    sort .t > got
    if ! diff -u exp got; then
      echo "*** Lines lost in run $i"
      let lost_count++
    fi
  done

  echo
  echo "=== Runs where a line was lost: $lost_count/$((i-1))"


And here is an excerpt from the results:


  $ bash foo.bash--- exp        2013-05-23 23:43:26.179397966 +0200
  +++ got       2013-05-23 23:43:26.347400928 +0200
  @@ -4,7 +4,6 @@
   3
   4
   5
  -6
   7
   8
   9
  *** Lines lost in run 10
  ...
  --- exp       2013-05-23 23:43:26.179397966 +0200
  +++ got       2013-05-23 23:43:28.075431391 +0200
  @@ -2,7 +2,6 @@
   1
   2
   3
  -4
   5
   6
   7
  *** Lines lost in run 137
  --- exp       2013-05-23 23:43:26.179397966 +0200
  +++ got       2013-05-23 23:43:28.255434565 +0200
  @@ -2,7 +2,6 @@
   1
   2
   3
  -4
   5
   6
   7
  *** Lines lost in run 148

  === Runs where a line was lost: 40/150


Few details about the system where I obtained the above:

  $ uname -rsv
  Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2
  $ /lib/x86_64-linux-gnu/libc.so.6
  GNU C Library (Debian EGLIBC 2.17-3) stable release version 2.17, by Roland 
McGrath et al.
  Copyright (C) 2012 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions.
  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
  PARTICULAR PURPOSE.
  Compiled by GNU CC version 4.7.3.
  Compiled on a Linux 3.8.12 system on 2013-05-15.
  Available extensions:
        crypt add-on version 2.1 by Michael Glad and others
        GNU Libidn by Simon Josefsson
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
  libc ABIs: UNIQUE IFUNC


BTW, the reported issue is quite old:

  <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11413>
  <http://lists.gnu.org/archive/html/automake/2012-08/msg00000.html>
  <http://lists.gnu.org/archive/html/automake-patches/2009-03/msg00064.html>

and the following suggests it might not be easy to fix:

  <http://lists.gnu.org/archive/html/automake-patches/2009-03/msg00073.html>

But it's worth nothing that the issue is not present with FreeBSD make (as
offered by Debian package freebsd-buildutils 9.0-11); maybe the sources of
that package might suggest how to obtain a fix after all?

Regards,
  Stefano