Re: CPU stress tests for LilyPond

[David Wright]

On Thu 02 Dec 2021 at 09:05:08 (+0100), Thomas Scharkowski wrote:
> On Thu, Dec 2, 2021 at 2:41 AM Paolo Prete wrote:
> > On Tue, Nov 30, 2021 at 5:56 PM Hans Åberg wrote:
> > > On 30 Nov 2021, at 16:36, Jacques Menu wrote:

> the Carver takes 10.6“ to compile on my MacBook Pro M1 16GB RAM with LilyPond 
> 2.23.5

That seems a lot more realistic than:

> > > > Here is what I get for this 102 page score:
> > > > 
> > > > real  0m39.590s
> > > > user  0m37.119s
> > > > sys   0m2.285s
> > > > 
> > > > The machine is a Mac Mini M1, 8GB RAM, 256 GB disk.

> > I don't think this test can give reliable results for what we need to 
> > compare.

Well, I think it's important give attention to reproducibility.
For example, here are some run results on Carver, converted to
2.23.3, which produces a 114 page score of 1059480 bytes. They're
all Intel processors running buster.

vintage   geekbench        1st run          2nd run
 2013        731             n/a             23/22
 2011        541            47/33            29/28
 2006       ~190           104/82            75/72

Times are real/user secs. (system makes up the difference).
n/a because this machine has run LP countless times before,
including this source. The other machines had not seen this
LP version or the Carver source before.

Looking at the product of benchmark*seconds, they're fairly
consistent: 16813, 15689, 14250 respectively over a range
of times of about 3½.

Your result yields a product of 18083, which would imply that
the benchmark is a little optimistic, as machines get faster,
for LP compilations, which agrees with Paolo:

> > From what I see,  is it true that geekbench 5 is single core, but it 
> > intensively uses RAM and the test we need should not be focused on RAM.

There are a lot of other factors that have to be weighed,
which might be one reason why these benchmarks look at
many determinations and their distribution. For example,
a power-saving CPU regime can slow down the processor,
and a judicious choice of video card and driver can speed
things up.

But one other comparison, between two 2.18.2 versions (the
website download and Debian's buster) on the 731 PC shows
the overriding importance of comparing like with like:

Version              2nd run  PDF bytes
Website download      76/58    1107477
Debian's buster       32/32    1084366

I assume the difference in size reflects the warning message
from buster's version (context was previously attached):
warning: do not know how to embed "C059-Italic"=
         "/usr/share/fonts/type1/urw-base35/C059-Italic.t1"

But the speed factor of 2:1 is remarkable. Any suggestions?

> > > So, to illustrate the idea of using the benchmark at the list below, it 
> > > might be the Mac mini (Late 2020) with single-core score 1712 and the 
> > > other Macs of this year have a similar performance. It gives for the iMac 
> > > 2008 used a single-core score 372 (depending on model), and it took 4–5 
> > > minutes to compile the same example. Then 1712/407 = 4.6, and dividing 4 
> > > minutes with that gives 53 seconds, and this ignores speedups in lilypond 
> > > self, but it seems one can get a rough idea of performance this way.

I can't understand the slowness of the iMac 2008, which I would
expect to take of a little under a minute. My first advice is to
try two consecutive runs to see if it speed up somewhat.

> > With my processor (Intel Celeron N3350) it took 96 seconds to compile 
> > MSDN.ly 
> > 
> > But my CPU is listed with score 287:

That too seems slow. What OS is it running?

> > https://openbenchmarking.org/vs/Processor/Apple%20M1,Intel%20Celeron%20J3455
> > 
> > (note that it compares Apple M1 with Celeron J3455, which is NOT my CPU, 
> > but it should be very similar for single-core tests.)
> > 
> > The result is 91 / 40 which is very similar to 96 seconds / 40 seconds 
> > (Jacque's test)  . I don't know if this is a coincidence and please, if you 
> > all have more data, share it.

I can't get results from that site because only the
fastest machine above matches: there's nothing comparable
even to the middle one, and the slowest probably raises
some eyebrows.

Cheers,
David.