Re: [RFC]serialize the output of parallel make?

[Edward Welbourne]

>>> This seems like quite an extreme example. stdout is line buffered by
>>> default,

on half-way decent systems - and even then, I'm not sure, it might be
limited to when writing to a TTY.

>> I use "make -j 4" to build and test gcc, the situation above is very common.

> Then it means you're getting a lot of diagnostics written to stderr, and you 
> should probably look into why you're getting so many.

While I can agree as concerns output while building gcc (at least
after the first pass, so you're using a half-way decent compiler), I
know we've had this problem with a certain compiler a certain
consortium insists on using (so we have to use to get our product on
their platform).  That compiler produces megabytes of warnings per
build, mostly spurious (including - joy - some warnings against code
that isn't actually present in the source; it's actually warning
against the code *it generated* to implement bit-field initializers in
C++ constructors <slap forehead="own" />) or, at the very least,
uninteresting - but enough of them need to be checked up (and some of
them are Important, albeit only on that platform, due to deficiencies
of its tool-chain) that we need to have a parser (because we sure
aren't going to read those megabytes ourselves) check the output and
tell us the interesting bits.

We're running GNU make under cygwin to drive this monstrosity (of
course, it's a windows-only development environment - the sort of
consortium that's going to impose such brain-dead-ness on ISVs isn't
likely to believe in other platforms): and when we turned on -j2
(because we'd finally moved the painfully slow auto-builder to a
machine with enough cores to make that useful; we can now use -j4 but
our first test-builds were cautious) our parser promptly broke because
we got interleaved output.  My previously-posted hack was the
work-around for that.

While large amounts of diagonostics from a sensible compiler like gcc
are indeed a good reason to *fix your damn code* so that interleaving
of diagnostics isn't an issue (and this is exactly what we've done on
platforms where we use gcc), there are (lamentably) times when one is
obliged to use less sensible software to earn an honest crust.  If
that spams you with diagnostics, you need to be able to at least
ensure they reach you in a clean enough form to support automatic
sorting of the wheat from the chaff.

Of course, that doesn't mean that the serialization *has* to be done
by make - I *can* hack round the problem and I'm pleased to hear there
are other tools to help - but it remains this problem is likely to be
prevalent among users of make -j, which *is* a case for make to
provide build-in support for it, if it's practical and someone is
willing to do the work to implement it,

        Eddy.