[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Chicken-users] two minor tweaks to runtime.c

From: Alaric Snell-Pym
Subject: Re: [Chicken-users] two minor tweaks to runtime.c
Date: Thu, 29 Sep 2011 14:09:40 +0100
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110617 Thunderbird/3.1.11

Hash: SHA1

On 09/29/2011 01:44 PM, Jörg F. Wittenberger wrote:
> On Sep 29 2011, Alaric Snell-Pym wrote:
>> On 09/29/2011 12:38 PM, Jörg F. Wittenberger wrote:
>>> I don't not have benchmarks for a reason: they would cost me too much
>>> time to do right.  Personally I don't believe too much in benchmarks
>>> anyway.  I believe in fast execution and source code review.
>> Ah, but how can you measure fast execution without a benchmark?
> Well, at the end of the day I run some complex applications.
> For instance I use httperf.

Then that, my good man, is your benchmark :-)

Yes, it's complex to run a chicken-based app and then run httperf
against that to get real-world performance figures. Ideally you'd be
able to distill the essence of that a little into a simpler test that
nonetheless had almost the same performance characteristics. And maybe
make mistakes in the process - but that's the art of performance estimation.

>> If the supposed performance improvement can't be benchmarked, then it's
>> pointless,
> Hey, I wrote "crawling slow"!  That's worse than just a benchmark.
> If the same program needs seconds instead of a fraction, then I don't
> bother.  I tracked the problem down, fixed a single thing and found
> a huge speed up.  As expected.  Case was closed for me: goal reached.

If you had some operation you could measure before and after to see that
speed up, then that's a benchmark too!

> At that time there was no point in perfomance measurements.

If you found a huge speed up, you were measuring performance, by
definition :-)

> If something works better for me than the current state of affairs,
> then I notify the list.  Code review is due upon integration anyway.
> And those who do have benchmarks already are welcome to post their
> results.  Reviews are welcome to challenge the code.

For sure. I, for one, think it's great that you're contributing
performance patches. But they do need to be checked with actual
benchmarks. It looks like you *have* been doing that, but claiming you
haven't, due to a disagreement about the definition of "benchmark" ;-)

Case in point: In the project I work on for a living, I have been
working on a series of performance improvements lately. In profiling, I
found that a critical system thread spent a large fraction of its time
in a function that was trivially optimisable; it was doing an
inter-process call back into the same process (don't ask why...), which
was then queued amongst other incoming requests before being handled,
and the handler then simply checked the existence of a certain file, and
sent the results back... so I changed the function to check for the
existence of the file itself, without the IPC back into the same process.

Result? Performance roughly HALVED. This was a surprise, but not a great
shock, as the performance of complex threaded software is famous for
being unexpected. We didn't have time to properly investigate, but I
believe the problem is that causing this operation to run faster causes
the thread to spend more of its time doing other things (rather than
blocking on the IPC), and is therefore holding certain locks for a
greater percentage of the time, blocking out other, more useful, things
that could be happening.

But without a benchmark, I'd have thought that this change couldn't
possibly have any other effect than to improve performance :-)

> So we are back at the capacity issue again: I have only so much live
> time.  I'm not ready to spent it maintaining benchmarks in the hope
> that those will eventually convince someone to benefit from the work I did.
> You are free to convince yourself.

That's fine; it's just that you were complaining you didn't want to do
benchmarks, which I now think was purely a communications mistake. Your
httperf results are benchmarks, too, if a little informal, but
infinitely better than just making changes that through inspection
"should" improve performance, which was my fear :-)

> /Jörg


- --
Alaric Snell-Pym
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla -


reply via email to

[Prev in Thread] Current Thread [Next in Thread]