RE: AVR Benchmark Test Suite [was: RE: [avr-gcc-list] GCC-AVR Registeroptimisations]

[Weddington, Eric]

> -----Original Message-----
> From: 
> address@hidden 
> [mailto:address@hidden
> org] On Behalf Of John Regehr
> Sent: Sunday, January 13, 2008 11:10 AM
> To: address@hidden
> Subject: Re: AVR Benchmark Test Suite [was: RE: 
> [avr-gcc-list] GCC-AVR Registeroptimisations]
> 
> I'll just plug Avrora again:
> 
>   http://compilers.cs.ucla.edu/avrora/
> 
> It runs on many platforms (written all in Java), is quite 
> fast, and is 
> well designed.  Best of all it is easy to extend, you just 
> add "monitors" 
> that can be configured to receive a wide variety of callbacks about 
> program events such as memory operations, I/O operations, 
> execution of 
> different kinds of instructions, interrupts, etc.

I'm definitely open to it. I just don't have much experience with it.
 
> > focus
> > on AVR-specific code, and GCC-specific AVR code at that.
> 
> Definitely.  If people want to test avr-gcc against other 
> compilers, or 
> compare AVR to other architectures, that's a separate exercise.

Sure. I'm also open to adding things in stages.
 
> MiBench is an aging but useful collection of embedded C codes:
> 
>   http://www.eecs.umich.edu/mibench/
> 
> > John, I would welcome publicly available code from TinyOS, 
> but I would
> > need to be already compiled with nesc, so that way we just 
> have straight
> > C that we can feed into avr-gcc.
> 
> Sure, this is easy.  It'll target ATmega128 only, howver.


I'm ok with that. We have to start somewhere.

 
> Re. floating point I believe that the "papabench" codes do a 
> lot of this:
> 
> http://www.irit.fr/recherches/ARCHI/MARCH/rubrique.php3?id_rubrique=97
> 
> This is code extracted from the Paparazzi UAV project, which uses an 
> ATmega for onboard flight control.
> 
> > There needs to be some consensus on what we measure, how we 
> measure it,
> > what output files we want generated, and hopefully some way to
> > automatically generate composite results. I'm certainly 
> open to anything
> > in this area.
> 
> Code size and static RAM consumption are obvious.  Some sort 
> of throughput 
> metric is useful.  For interrupt-driven codes, my group often uses 
> processor duty cycle as a measure of efficiency.  This is the 
> % of time 
> that the CPU is not in a sleep mode.  

Sure, that makes sense for wireless sensor networks (WSNs). But those
applications typically ensure that they go to sleep (for longer battery
life). We can't always expect that of other applications.

> Dyanmic stack memory 
> consumption is 
> good, though this is not a very consistent metric for 
> interrupt-driven 
> codes since in a short simulation run the worst-case stack usage is 
> unlikely to be encountered.  Perhaps adding up the stack 
> memory usage of 
> main + all interrupts would be better.

Oh, hey! I've heard of good tool for this ... It analyzes the
application statically... What was it called?
Oh, yeah, stacktool! ;-) Maybe we could use this... :-)

Eric W.