Re: Extracting PC information from QEMU/KVM during single-step

[Steven Raasch]

Alex -

Our application is similar to a high-end game in that it adjusts it's code-path depending on how "fast" it is running. Our previous attempts to trace it's execution have run into issues due to the overhead of the trace collection.

My hope had been that by using QEMU, I could control the "hardware" that the app is running on such that the app doesn't know that it's not running at full speed. In my mind, this means instrumenting the VM itself.

Using GDB is my second choice, as I'm not sure that I can gather the trace w/out slowing down the app's execution. I haven't dug into how GDB interacts with QEMU and GDB-scripting as of yet.

-Steve

On Wed, Jun 23, 2021 at 8:46 PM Alexander Bulekov <alxndr@bu.edu> wrote:

On 210623 1514, Steven Raasch wrote:
> Hi -
>
> I'm trying to create a hack that will allow me to extract an instruction
> trace from QEMU/KVM (i386). The KVM part is important (see below).
>
> Background:
>
>    - I have used KVM to create a snapshot of a windows-10 guest running a
>    graphics-intensive app. The *original* issue is that the app does not
>    execute correctly when re-started from the snapshot using TCG (it doesn't
>    crash, but it doesn't run correctly, either). So, using the existing "-d
>    in_asm" won't work. It seemed to me that hacking in tracing to KVM should
>    be easier than figuring out why the app doesn't work.
>    - I've poked around the tracing mechanism in the TCG, and extracted what
>    I need to dump instructions and then added that to kvm_cpu_exec().
>    - I'm setting DEBUG & single-step modes by calling cpu_single_step()
>    from the top of kvm_vcpu_thread_fn().
>    - in kvm_cpu_exec() I wait until I get a KVM_EXIT_DEBUG signal before
>    logging the instruction.
>
> I have the output of TCG "-d in_asm" from the beginning of the execution,
> and I'm comparing the KVM output with that.
>
> What I don't have right is the PC of the instruction that's been executed.
> The TCG is clearly sane, but the KVM output is not.
>
> My best thought was to extract the PC from kvm_run (run->debug.arch.pc)
> after the KVM_RUN ioctl, but that doesn't match up. I also tried
> kvm_vcpu_ioctl() with KVM_GET_REGS, and grabbing the rip from cpu->env.rip.
> I didn't expect any of these to be *exactly* right, but I thought they
> would lead me to something sane.
>
> Using run->debug.arch.pc gives me the right address for the first
> instruction, but nothing makes sense after that.
>
> Can anyone help me get onto the right track?
>
> Thanks!
>
> -Steve

Is there some reason you can't do this using qemu's gdbstub and gdb? It
supports single-stepping under KVM.

-Alex