Re: ftbench: last updates

[Werner LEMBERG]

Hello Ahmet,


> -I have changed the * and the sentence

Thanks.  Unfortunately, I was unclear that '*x*' in my e-mail is meant
as Markdown syntax and not to be taken verbatim.  In other words, 'x'
should be typeset in italics, similar to a mathematical variable.
Sorry for that, and please fix.

Something different: Please don't use `\r` in messages for newlines.
Use `\n` instead, which is standard (file `testing.mk`).

>> I already changed the working way of the timing. I only start the
>> benchmark at beginning and stop at the end.
>
> i mean, it times chunks, not single iteration.  timer starts at the
> beginning of the chunk and stop at the end (then divide the results
> size of a chunk). because of it does not time single iteration, it
> is already a bulk test.

Ok.

>> BTW, I suggest that you add another sentence, explaining *why*
>> there are two values at all.
>
> actually, i didnt get the reason well but it may differ even with
> same flags. i need help in this case.

Alexei, please have a look.  It seems that only(?) the 'Stroke' test
is affected...

> the code seems producing more accurate results after splitting the
> results into chunks. are results seem satisfactory in your machine?

Alas, they aren't.  Running your code on my GNU/Linux machine, I get
percental differences up to 14%.

BTW, I've also just compiled `valgrind` from its git repository (to be
sure that it exactly fits my 'libc' library version) and run

```
valgrind --trace-children=yes make baseline
valgrind --trace-children=yes make benchmark
```

expecting to get identical results, because `valgrind` uses a CPU
emulator.  However, to my great surprise, they are *not* identical,
and sometimes the values differ even enormously.  Has anybody an idea
why this happens?  AFAICS from valgrind's source files, the
`clock_gettime` function *is* caught.

Note that with valgrind the creation of the webpage fails; I get

```
Traceback (most recent call last):
  File "/home/wl/ahmet/tohtml.py", line 310, in <module>
    main()
  File "/home/wl/ahmet/tohtml.py", line 67, in main
    generate_total_results_table(html_file, BASELINE_DIR, BENCHMARK_DIR)
  File "/home/wl/ahmet/tohtml.py", line 192, in generate_total_results_table
    diff = ((baseline - benchmark) / baseline) * 100
ZeroDivisionError: float division by zero
```

Attached are the two directories from valgrind runs.  Comparing this
with non-valgrind values I think there is a fundamental interaction
problem with the `bench` program: It seems that `bench` controls the
number of loops *in addition* to what you specify via the command
line.  Unfortunately, I don't have time right now to check this by
myself.

Too bad that you didn't try `valgrind` by yourself earlier, as I've
asked you repeatedly to do ...

> Also, While there are less than 10 days for final evaluation, there
> are points that are not completed:
>
> * meson
> * cmake
> * documentation
>
> because of our focus a bit changed, didnt worked on them
> much. Should I complete them all? Is there a priority?

Please forget meson and cmake, this is not important at this stage.
What I now ask is to create a new branch, with *all* of your changes
recreated as logical and (relatively small) incremental commits,
rebased to 'master'.  Ideally every commit is compilable.  If not
(which should be an exception, and there should be a good explanation
for that), please add a remark in the commit message.

* Try to adjust the formatting to be as near to the remaining code of
  FreeType.

* The commit messages should also be in FreeType style, which is much*
  *more verbose than what you currently do.

* Avoid trailing whitespace – whatever editor you use, *please* change
  the settings so that this gets automatically removed.  Right now, I
  see zillions of trailing whitespace in file `tohtml.py`.

* Hin-Tak is right: Documentation is of paramount importance so that
  you (or someone else) can continue the work.

Have a look at previous GSoC students' (remote) branches to see how
such a recreated branch should look like, for example
`GSoC-2019-nikhil` or `GSoC-2017-ewaldhew`.


    Werner

valgrind.tar.gz
Description: Binary data