Re: Weird performance problem

[Leandro Lucarella]

Greg Chicares, el 14 de julio a las 17:27 me escribiste:
> On 2010-07-14 14:52Z, Leandro Lucarella wrote:
> > Long story short, I have a program that when executed from a shell it
> > takes about 3 seconds to complete, but when executed via make, it takes
> > 5 seconds.
> 
> I trivially modified your very clear testcase here:
> 
> > http://www.llucax.com.ar/blog/blog/post/-7a56a111
> 
> to run a unit-test program of my own, but didn't see any penalty:
> 
> $echo 'all:' > Makefile
> $echo -e '\t$C' >> Makefile
> $make C="./elapsed_time ./zero_test-a"
> ./elapsed_time ./zero_test -a
> Elapsed time: 43 milliseconds
> $./elapsed_time ./zero_test -a
> Elapsed time: 43 milliseconds

Note that the problem is not happening with any program, just with the
voronoi test. So I'm not surprised that you didn't see the effects with
your custom test program.

> > Is there anything make does to the environment (in a general sense, not
> > only environment variables) before executing commands that could cause
> > a difference like this?
> 
> To diagnose this further, you might try echoing the precise time
>  - before you start 'make'
>  - just before running your test program
>  - just after  running your test program
>  - right after 'make' ends

The time was measured by several methods, including from inside the
program (option -v) and using strace to trace both the program and make,
and the difference are not because of make, the same program takes
longer to execute itself when ran from make. Also note that the program
without a change the change in the post runs equally fast inside and
outside make. I can understand a program having different performances
with and without a change, but I don't understand why is that difference
only showing when executed inside make (and the difference is so
reproducible and stable, even among different hardware).

> My first guess would be that the difference you observe is shell
> startup time.

Nope, that can't be it, because the time measured from inside the
program is still 2 seconds bigger.

> I realize that you have evidence that argues against
> that (because only one test program seems to be affected). But you
> have already measured running your program directly:
>   /usr/bin/time -f%e ./voronoi -n 30000
> and it wouldn't hurt to run
>   sh -c '/usr/bin/time -f%e ./voronoi -n 30000'
> just to see whether that takes remarkably longer.

As I said before, I even tried measuring the time inside the program
(the code to do that is already there, with the option -v).

> If you have defined a $SHELL variable in the environment, 'make'
> will use that instead of '/bin/sh'.

I even tried to use different shells inside make, no changes.

I know is really odd and if it didn't happened to me, I probably
wouldn't believe it =)

BTW, I can publish a compiled version of the programs (before and after
the change that trigger this behaviour) if you want to do any testing,
I know that compiling it from the sources is really hard since it's
written in the D programming language and using a custom runtime I'm
working on.


Thanks for your answer.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Hey you, out there beyond the wall,
Breaking bottles in the hall,
Can you help me?