Re: GNU make 3.81beta4 released

[Markus Mauhart]

"Eli Zaretskii" <address@hidden> wrote ...
>
>> From: "Markus Mauhart" <address@hidden>
>> Date: Thu, 19 Jan 2006 00:55:46 +0100
>>
>> Now I continued with your suggestion "-j 64" -- it run AFAICS allmost
>> 1m without errors until I got ...
>>     Assertion failed: a == g->changed, file .\remake.c, line 169
>> ... this comes from an assertion I had inserted around this bug:
>>     g->changed += commands_started - ocommands_started;
>> (g->changed is only 8 bits wide).
>
> What is the value of commands_started and ocommands_started at that
> point, and what is the value of g->changed?

see today's 3 traces below

> What exactly is that assertion testing?  I mean, what is `a' to which
> it compares the value of g->changed?

yesterday it was ...

{
int a = (int)commands_started - (int)ocommands_started;
a += g->changed;
g->changed += commands_started - ocommands_started;
assert (a == g->changed);
}

Today I changed it to ...

{
unsigned int const g_changed_old = g->changed;
int a = (int)commands_started - (int)ocommands_started;
a += g->changed;
g->changed += commands_started - ocommands_started;
if (a != g->changed)
    {
    fprintf (stderr, "\n(g_changed_old, commands_started, ocommands_started) = 
(%u ,%u ,%u)\n"
            ,g_changed_old ,commands_started ,ocommands_started);
    g->changed = (commands_started != ocommands_started);
    }
}

3 traces from 3 successfull(!) "-j 64" runs:

(g_changed_old, commands_started, ocommands_started) = (0 ,419) //forgot one %u
(g_changed_old, commands_started, ocommands_started) = (0 ,436) //forgot to 
rebuild
(g_changed_old, commands_started, ocommands_started) = (0 ,466 ,0)

IOW, cause I usually dont have multiples of 256, also the original code would 
work,
especially yesterday's "-j 64"-failure was probably caused by the assertion.

Btw, such successfull "-j 64"-build then sometimes uses ~50 compilers
in parallel - with my 1GB this is no problem with msvc71, while yet "-j 3" is
a big problem with mingw-gcc344 (both -O2).


> Anyway, it looks like g->changed is just a boolean flag, its value is
> tested in remake.c to be either zero or non-zero.  So, unless I'm
> missing something, the offending line can be modified to say:
>
>    g->changed = commands_started - ocommands_started;

or ... = (commands_started != ocommands_started)


So 2 issues from my tests remain open:

1) "-j someNumber" propagates to recursive $(MAKE) only via a (simple) trick,
but this trick also disables gmake's builtin "load balancing" for tree's
of sub-MAKE's (not a problem for my test's simple tree with on trivial root
and exactly one child).


2) "-j noNumber" (btw, array sizes surely reduced to MAXIMUM_WAIT_OBJECTS, and
                 overflows handeled correctly for some levels of callers)
As I said, this has the advantage of propagating to recursive $(MAKE) whithout
tricks, but the disadvantage of looping forever in all my tests:
Initially ~30 trivial cmds (.bat files) are executed successfully (containing 
echo ...).
Then in my last test I get 89 traces ...
    process_easy() failed failed to launch process (e=-1)
    // job.c contains the double "failed" typo
... only interrupted with a single (trivial) success.
Then some 100 successfull cmds (echos and compiles and links).
Then it loops forever.
89 .bat-files still left in the tmp dir.
Many (89 ?) childs existing.
IMHO there are to bugs involved:
1st, the failed process_easy() isnt really gracefully handled - the 
"transactions"
arent undone and hence cant be re-tried.
2nd, the loop and its sub-functions fails to detect "nothing goes":
E.g. in reap_children() "if (!remote && pid == shell_function_pid)" in my case
is "true" cause both pid and shell_function_pid are 0, but the corresponding
code and comment IMHO acts like handling lots of success.
Both probably are part of the problem you mentioned IIRC that generally -j isnt
safe against temporary resource allocation failures.

Tomorrow evening or Saturday morning I'll post the necessary URLs and info to
reproduce the tests I did. I'm currently using mingw-gcc and MSVC inside 
cmd.exe.
GCC inside Cygwin's bash is currently seriously degraded cause the sources I use
recently incorporated use of xxx64-functions not available in my cygwin headers.
The sources do build under linux, out-of-the box with their corresponding
integrated builder "TheIDE". To build with my makefile ("TheMAKE" :-) under
linux I guess some minor bugfixes in their cfg-files will be necessary (as it
was necessary with xxx64-less sources under Cygwin).


Best Regards,
Markus.