Hi -
I've made some decent progress with rpctrace this week. I think I pretty much understand this program now!
In particular, I've gotten it working on ext2fs. I had to fix the problem I described earlier with send-once rights, and then deal with a deadlock situation caused when ext2fs called vm_copy() on a region of memory that it itself managed. This caused the kernel to do some memory manager RPCs before the vm_copy() completed - the same kind of problem I described two months ago with netmsg. My solution is to make rpctrace multi-threaded. It now has a global lock that protects pretty much the entire program, but gets unlocked right before resending a message, which prevents the deadlock. I've also added a condition variable to ensure that messages get processed in-order, even if two different threads attempt to process two messages in the wrong order.
So, ext2fs can now be traced. I'm seeing some bizarre behavior with it, running on a little ramdisk. For example, when being traced, for some reason it doesn't use the memory map interface, but instead does everything with io_read/io_write, then doesn't perform the last io_write when being detached and leaves the ramdisk in a dirty state. I've also seem some kernel panics in gsync_wait().
All in all, I think it's enough progress to warrant a patch set, even though I've still got more stuff to investigate here.