Re: ftbench: last updates

On 18 Sep 2023 10:34 +0300, Werner LEMBERG <wl@gnu.org>, wrote:

Hello Ahmet,

-I have changed the * and the sentence

Thanks. Unfortunately, I was unclear that '*x*' in my e-mail is meant
as Markdown syntax and not to be taken verbatim. In other words, 'x'
should be typeset in italics, similar to a mathematical variable.
Sorry for that, and please fix.

Something different: Please don't use `\r` in messages for newlines.
Use `\n` instead, which is standard (file `testing.mk`).

I already changed the working way of the timing. I only start the
benchmark at beginning and stop at the end.

i mean, it times chunks, not single iteration. timer starts at the
beginning of the chunk and stop at the end (then divide the results
size of a chunk). because of it does not time single iteration, it
is already a bulk test.

Ok.

BTW, I suggest that you add another sentence, explaining *why*
there are two values at all.

actually, i didnt get the reason well but it may differ even with
same flags. i need help in this case.

Alexei, please have a look. It seems that only(?) the 'Stroke' test
is affected...

the code seems producing more accurate results after splitting the
results into chunks. are results seem satisfactory in your machine?

Alas, they aren't. Running your code on my GNU/Linux machine, I get
percental differences up to 14%.

BTW, I've also just compiled `valgrind` from its git repository (to be
sure that it exactly fits my 'libc' library version) and run

```
valgrind --trace-children=yes make baseline
valgrind --trace-children=yes make benchmark
```

expecting to get identical results, because `valgrind` uses a CPU
emulator. However, to my great surprise, they are *not* identical,
and sometimes the values differ even enormously. Has anybody an idea
why this happens? AFAICS from valgrind's source files, the
`clock_gettime` function *is* caught.

Note that with valgrind the creation of the webpage fails; I get

```
Traceback (most recent call last):
File "/home/wl/ahmet/tohtml.py", line 310, in <module>
main()
File "/home/wl/ahmet/tohtml.py", line 67, in main
generate_total_results_table(html_file, BASELINE_DIR, BENCHMARK_DIR)
File "/home/wl/ahmet/tohtml.py", line 192, in generate_total_results_table
diff = ((baseline - benchmark) / baseline) * 100
ZeroDivisionError: float division by zero
```

Attached are the two directories from valgrind runs. Comparing this
with non-valgrind values I think there is a fundamental interaction
problem with the `bench` program: It seems that `bench` controls the
number of loops *in addition* to what you specify via the command
line. Unfortunately, I don't have time right now to check this by
myself.

Too bad that you didn't try `valgrind` by yourself earlier, as I've
asked you repeatedly to do ...

Also, While there are less than 10 days for final evaluation, there
are points that are not completed:

* meson
* cmake
* documentation

because of our focus a bit changed, didnt worked on them
much. Should I complete them all? Is there a priority?

Please forget meson and cmake, this is not important at this stage.
What I now ask is to create a new branch, with *all* of your changes
recreated as logical and (relatively small) incremental commits,
rebased to 'master'. Ideally every commit is compilable. If not
(which should be an exception, and there should be a good explanation
for that), please add a remark in the commit message.

* Try to adjust the formatting to be as near to the remaining code of
FreeType.

* The commit messages should also be in FreeType style, which is much*
*more verbose than what you currently do.

* Avoid trailing whitespace – whatever editor you use, *please* change
the settings so that this gets automatically removed. Right now, I
see zillions of trailing whitespace in file `tohtml.py`.

* Hin-Tak is right: Documentation is of paramount importance so that
you (or someone else) can continue the work.

Have a look at previous GSoC students' (remote) branches to see how
such a recreated branch should look like, for example
`GSoC-2019-nikhil` or `GSoC-2017-ewaldhew`.

Werner

Info	Baseline	Benchmark
Parameters	-c 1000 -w 100	-c 1000 -w 100
Commit ID	e61380fe	e61380fe
Commit Date	2023-09-21 15:34:38 +0300	2023-09-21 15:34:38 +0300
Branch	gsoc-2023-ahmet-final	gsoc-2023-ahmet-final

* If a value in the 'Iterations' column is given as 'x | y', values x and y give the number of iterations in the baseline and the benchmark test, respectively.

From:	Ahmet Göksu
Subject:	Re: ftbench: last updates
Date:	Thu, 21 Sep 2023 23:54:31 +0300

Test	Iterations	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	100000	7.2	7.0	2.4
Load_Advances (Normal)	100000	6.4	6.2	1.7
Load_Advances (Fast)	100000	0.0	0.0	-4.8
Load_Advances (Unscaled)	100000	0.0	0.0	-0.7
Render	100000	9.7	9.3	4.1
Get_Glyph	100000	7.0	6.8	3.2
Get_Char_Index	94000	0.0	0.0	3.0
Iterate CMap	1000	2.1	2.0	4.3
New_Face	1000	51.8	49.1	5.3
Embolden	100000	7.8	7.6	3.4
Stroke	56642 \| 57701	36.3	35.6	1.9
Get_BBox	100000	5.8	5.7	1.3
Get_CBox	100000	7.0	6.8	1.6
New_Face & load glyph(s)	100000	1.1	1.1	0.6
Total duration for all tests:	81 s

Test	Iterations	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	120000	4.9	4.9	0.6
Load_Advances (Normal)	120000	4.2	4.2	0.6
Load_Advances (Fast)	120000	0.0	0.0	-21.4
Load_Advances (Unscaled)	120000	0.0	0.0	-7.7
Render	120000	7.6	7.4	2.9
Get_Glyph	120000	4.9	4.9	-0.2
Get_Char_Index	94000	0.0	0.0	0.0
Iterate CMap	1000	2.0	2.0	0.3
New_Face	1000	43.5	42.8	1.5
Embolden	120000	5.3	5.3	0.6
Stroke	60720 \| 60600	33.0	33.0	-0.1
Get_BBox	120000	1.2	1.2	-2.2
Get_CBox	120000	5.0	4.9	1.3
New_Face & load glyph(s)	120000	0.9	0.9	1.2
Total duration for the font:	14 s

Test	Iterations	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	9.7	8.4	12.9
Load_Advances (Normal)	95000	8.7	7.9	9.1
Load_Advances (Fast)	95000	0.0	0.0	0.0
Load_Advances (Unscaled)	95000	0.0	0.0	0.0
Render	95000	11.9	10.6	10.7
Get_Glyph	95000	9.2	8.1	12.1
Get_Char_Index	94000	0.0	0.0	10.0
Iterate CMap	1000	2.3	2.0	9.8
New_Face	1000	62.2	52.2	16.1
Embolden	95000	10.1	8.6	15.1
Stroke	51585 \| 55670	38.8	36.0	7.4
Get_BBox	95000	7.9	7.5	4.3
Get_CBox	95000	8.6	8.0	6.3
New_Face & load glyph(s)	95000	1.2	1.2	0.8
Total duration for the font:	19 s

Test	Iterations	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	9.9	10.0	-1.0
Load_Advances (Normal)	95000	9.0	8.9	0.6
Load_Advances (Fast)	95000	0.0	0.0	-3.6
Load_Advances (Unscaled)	95000	0.0	0.0	-7.7
Render	95000	12.9	12.6	2.2
Get_Glyph	95000	9.8	10.0	-1.3
Get_Char_Index	94000	0.0	0.0	0.0
Iterate CMap	1000	2.0	2.0	1.6
New_Face	1000	54.6	54.0	1.1
Embolden	95000	11.0	11.1	-0.4
Stroke	41895 \| 41990	47.8	47.7	0.2
Get_BBox	95000	9.6	9.6	0.2
Get_CBox	95000	9.9	10.0	-1.2
New_Face & load glyph(s)	95000	1.4	1.5	-1.1
Total duration for the font:	20 s

Re: ftbench: last updates

Freetype Benchmark Results

Warning: Baseline and Benchmark have the same commit ID!

Info

Total Results

Results for Roboto_subset.ttf

Results for Arial_subset.ttf

Results for TimesNewRoman_subset.ttf

Results for Tahoma_subset.ttf

Results for Verdana_subset.ttf

Test	Iterations	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	6.0	6.4	-6.8
Load_Advances (Normal)	95000	5.1	5.3	-2.8
Load_Advances (Fast)	95000	0.0	0.0	0.0
Load_Advances (Unscaled)	95000	0.0	0.0	0.0
Render	95000	8.3	8.1	1.7
Get_Glyph	95000	6.0	5.8	2.1
Get_Char_Index	94000	0.0	0.0	3.7
Iterate CMap	1000	2.0	2.0	2.8
New_Face	1000	51.9	49.5	4.5
Embolden	95000	6.7	6.6	1.8
Stroke	62985 \| 63935	31.8	31.3	1.5
Get_BBox	95000	5.4	5.3	0.4
Get_CBox	95000	5.9	5.8	1.2
New_Face & load glyph(s)	95000	1.1	1.1	1.6
Total duration for the font:	14 s