Questions about Disk I/O Record and Replay

[Arnabjyoti Kalita]

Hello all,

I am trying to use the blkreplay driver to record and replay disk
operations. My disk record and replay mechanism is a variant of the
inbuilt record replay mechanism in QEMU.

During record, I just store the completion ids of the disk operations
as they happen. Something like this -

void replay_block_event(QEMUBH *bh, uint64_t id)
{
    if (replay_mode == REPLAY_MODE_RECORD) {
        if (start_recording) {
            replay_put_qword(id, "disk");  // writes ids to a separate
file dedicated for disk I/O record
        }
        qemu_bh_schedule(bh);
    }
}

During replay, all I do is store the disk ID into an events list and
ask the CPU to continue executing instructions.

void replay_block_event(QEMUBH *bh, uint64_t id)
{
   if (replay_mode == REPLAY_MODE_RECORD) {
       /* as shown above */
   }
   if (replay_mode == REPLAY_MODE_PLAY) {
       BlockEvent *event = g_malloc0(sizeof(BlockEvent));
       event->opaque = bh;
       event->id = id;
       QTAILQ_INSERT_TAIL(&blk_events_list, event, blk_events);
       qemu_cpu_kick(first_cpu);                    <- Replayed guest
gets stuck here
   }
   else {
       qemu_bh_schedule(bh);
   }
}

I know when the disk interrupt happens and so all the event ids that
we store will be replayed right before that.

My questions are as follows -

1. Is the above approach a good idea to achieve deterministic disk I/O replay?

2. Should I also replay disk event I/O as and when they arrive? How do
I ensure that it actually gets completed before a checkpoint? The
checkpoint in this case, is the disk interrupt.

3. How can I handle out-of-order disk I/O completions, in this
scenario? How should I proceed?

I would love to get more details on these questions.

Thank you very much for all your help so far.

Best Regards,
Arnabjyoti Kalita