Re: [Qemu-devel] RFC: Design Doc for a new trace format (to support variable number/size of args per event) simpletrace-v2

[Stefan Hajnoczi]

On Tue, Nov 29, 2011 at 8:29 AM, Harsh Bora <address@hidden> wrote:
> Currently, Qemu provides an in-built "simple" trace backend which is simple
> and easy to use (no additional/external dependencies) and allows developers
> to trace events in Qemu code, however, it suffers from limitations like
> unability to trace more than 6 elements per trace event, lack of string
> support, etc. There are various places in Qemu code, where one would want to
> trace events having multiple arguments including strings. This results into
> motivation for defining an advanced, yet simple trace format which will
> address these limitations. For the sake of convinence, let us call this new
> trace format as simpletrace v2 (any other better name?).
>
> HLD for Simpletrace v2:
> ======================
>
> This new trace format defines 3 types of structures for trace data
> organization:
>
> 1) Trace Log Header (per log file, provides meta-data about the entire trace
> log, like trace format version, endianness, etc.)
> 2) Trace Event Header (per trace event, provides meta-data per trace event)
> 3) Trace Data (per argument/data in a trace event, provides size of data
> followed by data itself)
>
>
>  The Trace Log header can be defined like this:
>
> typedef struct {
>        uint64_t endian_magic;  /* =0xAA0011FF, a magic number helps
> identifying validity and endian-ness of trace log to its readers */
>        uint64_t pid;           /* pid of qemu process can be traced here */
>        uint64_t version;       /* Keeping version info as 3rd 64-bit element
> as expected by current simpletrace format */
>        unit64_t timestamp;     /* timestamp info */
> } TraceLogHeader;
>
> Suggestions are invited to make this header more informative as required.
>
> Further, this TraceLogHeader will be followed by 0 or more Trace Event
> Headers (further followed by trace data) as defined below:
>
> typedef struct {
>        uint64_t magic;         /* =0xA1B2C3D4, ensures a valid trace record,
> otherwise corrupted */
>        unit64_t id;            /* unique identifer per trace event */
>        uint64_t timestamp;     /* timestamp info */
>        uint64_t num_args;      /* number of arguments (followed by this
> TraceEventHeader) traced in this trace event */

What exactly is num_args?

> } TraceEventHeader;
>
> Trace Data is expected to be stored in following format followed by
> TraceEventHeader:
>
> typedef struct {
>        uint64_t size;  /* size of data in bytes to be read following this
> header */
>        uint8_t data[0] /* variable sized data */
> } TraceData;

This format does not support variable-length strings in a
self-describing way.  What I mean is that a trace record containing a
string should work in the simpletrace Python module:

class QMPAnalyzer(simpletrace.Analyzer):
    def handle_qmp_command(self, mon, cmd_name):
        print 'QMP command "%s" on monitor %#x' % (cmd_name, mon)

The cmd_name argument is a string and this script shows all QMP
commands that were handled.

A simple way of implementing this is to have simpletrace.py parse
trace-events and look for (const) char * arguments.  String arguments
are serialized like this:

uint16_t length;
char chars[length];

It may be useful to pad to the next 8-byte boundary after a string argument.

Stefan