Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#2] Dissecting QEMU Into Three Main Parts

[Ahmed Karaman]

On Tue, Jun 30, 2020 at 6:34 AM Lukáš Doktor <ldoktor@redhat.com> wrote:
>
> Dne 29. 06. 20 v 12:25 Ahmed Karaman napsal(a):
> > Hi,
> >
> > The second report of the TCG Continuous Benchmarking series builds
> > upon the QEMU performance metrics calculated in the previous report.
> > This report presents a method to dissect the number of instructions
> > executed by a QEMU invocation into three main phases:
> > - Code Generation
> > - JIT Execution
> > - Helpers Execution
> > It devises a Python script that automates this process.
> >
> > After that, the report presents an experiment for comparing the
> > output of running the script on 17 different targets. Many conclusions
> > can be drawn from the results and two of them are discussed in the
> > analysis section.
> >
> > Report link:
> > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Dissecting-QEMU-Into-Three-Main-Parts/
> >
> > Previous reports:
> > Report 1 - Measuring Basic Performance Metrics of QEMU:
> > https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06692.html
> >
> > Best regards,
> > Ahmed Karaman
>
> Hello Ahmed,
>
> very nice reading, both reports so far. One thing that could be better 
> displayed is the system you used this to generate. This would come handy 
> especially later when you move from examples to actual reports. I think it'd 
> make sense to add a section with a clear definition of the machine as well as 
> the operation system, qemu version and eventually other deps (like compiler, 
> flags, ...). For this report something like:
>
> architecture: x86_64
> cpu_codename: Kaby Lake
> cpu: i7-8650U
> ram: 32GB DDR4
> os: Fedora 32
> qemu: 470dd165d152ff7ceac61c7b71c2b89220b3aad7
> compiler: gcc-10.1.1-1.fc32.x86_64
> flags: 
> --target-list="x86_64-softmmu,ppc64-softmmu,aarch64-softmmu,s390x-softmmu,riscv64-softmmu"
>  --disable-werror --disable-sparse --enable-sdl --enable-kvm  
> --enable-vhost-net --enable-vhost-net --enable-attr  --enable-kvm  
> --enable-fdt   --enable-vnc --enable-seccomp 
> --block-drv-rw-whitelist="vmdk,null-aio,quorum,null-co,blkverify,file,nbd,raw,blkdebug,host_device,qed,nbd,iscsi,gluster,rbd,qcow2,throttle,copy-on-read"
>  --python=/usr/bin/python3 --enable-linux-io-uring
>
> would do. Maybe it'd be even a good idea to create a script to report this 
> basic set of information and add it after each of the perf scripts so people 
> don't forget to double-check the conditions, but others might disagree so 
> take this only as a suggestion.
>
> Regards,
> Lukáš
>
> PS: Automated cpu codenames, hosts OSes and such could be tricky, but one can 
> use other libraries or just best-effort-approach with fallback to "unknown" 
> to let people filling it manually or adding their branch to your script.
>
> Regards,
> Lukáš
>
Thanks Mr. Lukáš, I'm really glad you found both reports interesting.

Both reports are based on QEMU version 5.0.0, this wasn't mentioned in
the reports so thanks for the reminder. I'll add a short note about
that.

The used QEMU build is a very basic GCC build (created by just running
../configure in the build directory without any flags).

Regarding the detailed machine information (CPU, RAM ... etc), The two
reports introduce some concepts and methodologies that will produce
consistent results on whichever machine they are executed on. So I
think it's unnecessary to mention the detailed system information used
in the reports for now.

Best regards,
Ahmed Karaman