Re: [Qemu-devel] [PATCH 00/17] RFC: userfault v2

[zhanghailiang]

On 2014/10/31 13:17, Andres Lagar-Cavilla wrote:
> On Thu, Oct 30, 2014 at 9:38 PM, zhanghailiang
> <address@hidden> wrote:
> > On 2014/10/31 11:29, zhanghailiang wrote:
> > > On 2014/10/31 10:23, Peter Feiner wrote:
> > > > On Thu, Oct 30, 2014 at 07:31:48PM +0800, zhanghailiang wrote:
> > > > > On 2014/10/30 1:46, Andrea Arcangeli wrote:
> > > > > > On Mon, Oct 27, 2014 at 05:32:51PM +0800, zhanghailiang wrote:
> > > > > > > I want to confirm a question:
> > > > > > > Can we support distinguishing between writing and reading memory for
> > > > > > > userfault?
> > > > > > > That is, we can decide whether writing a page, reading a page or both
> > > > > > > trigger userfault.
> > > > > >
> > > > > > Mail is going to be long enough already so I'll just assume tracking
> > > > > > dirty memory in userland (instead of doing it in kernel) is worthy
> > > > > > feature to have here.
> > > >
> > > >
> > > > I'll open that can of worms :-)
> > > >
> > > > > [...]
> > > > > Er, maybe i didn't describe clearly. What i really need for live memory
> > > > > snapshot
> > > > > is only wrprotect fault, like kvm's dirty tracing mechanism, *only
> > > > > tracing write action*.
> > > > >
> > > > > So, what i need for userfault is supporting only wrprotect fault. i
> > > > > don't
> > > > > want to get notification for non present reading faults, it will
> > > > > influence
> > > > > VM's performance and the efficiency of doing snapshot.
> > > >
> > > >
> > > > Given that you do care about performance Zhanghailiang, I don't think
> > > > that a
> > > > userfault handler is a good place to track dirty memory. Every dirtying
> > > > write
> > > > will block on the userfault handler, which is an expensively slow
> > > > proposition
> > > > compared to an in-kernel approach.
> > >
> > > Agreed, but for doing live memory snapshot (VM is running when do
> > > snapsphot),
> > > we have to do this (block the write action), because we have to save the
> > > page before it
> > > is dirtied by writing action. This is the difference, compared to pre-copy
> > > migration.
> >
> > Again;) For snapshot, i don't use its dirty tracing ability, i just use it
> > to block write action,
> > and save page, and then i will remove its write protect.
>
> You could do a CoW in the kernel, post a notification, keep going, and
> expose an interface for user-space to mmap the preserved copy. Getting
> the life-cycle of the preserved page(s) right is tricky, but doable.
> Anyway, it's easy to hand-wave without knowing your specific
> requirements.

Yes, what i need is very much like user-space COW feature, but i don't want to 
modify
any code of kvm to relize COW, usefault is a more generic way and more grace.
Besides, I'm not an expert in kernel:(

> Opening the discussion a bit, this does look similar to the xen-access
> interface, in which a xen domain vcpu could be stopped in its tracks

Right;)

> while user-space was notified (and acknowledged) a variety of
> scenarios: page was written to, page was read from, vcpu is attempting
> to execute from page, etc. Very applicable to anti-viruses right away,
> for example you can enforce W^X properties on pages.
>
> I don't know that Andrea wants to open the game so broadly for
> userfault, and the code right now is very specific to triggering on
> pte_none(), but that's a nice reward down this road.

I hope he will consider it. IMHO, it is a good extension for userfault
(write fault);)

Best Regards,
zhanghailiang

> > > > > Also, i think this feature will benefit for migration of ivshmem and
> > > > > vhost-scsi
> > > > > which have no dirty-page-tracing now.
> > > >
> > > >
> > > > I do agree wholeheartedly with you here. Manually tracking non-guest
> > > > writes
> > > > adds to the complexity of device emulation code. A central fault-driven
> > > > means
> > > > for dirty tracking writes from the guest and host would be a welcome
> > > > simplification to implementing pre-copy migration. Indeed, that's exactly
> > > > what
> > > > I'm working on! I'm using the softdirty bit, which was introduced
> > > > recently for
> > > > CRIU migration, to replace the use of KVM's dirty logging and manual
> > > > dirty
> > > > tracking by the VMM during pre-copy migration. See
> > >
> > >
> > > Great! Do you plan to issue your patches to community? I mean is your work
> > > based on
> > > qemu? or an independent tool (CRIU migration?) for live-migration?
> > > Maybe i could fix the migration problem for ivshmem in qemu now,
> > > based on softdirty mechanism.
> > >
> > > > Documentation/vm/soft-dirty.txt and pagemap.txt in case you aren't
> > > > familiar. To
> > >
> > >
> > > I have read them cursorily, it is useful for pre-copy indeed. But it seems
> > > that
> > > it can not meet my need for snapshot.
> > >
> > > > make softdirty usable for live migration, I've added an API to atomically
> > > > test-and-clear the bit and write protect the page.
> > >
> > >
> > > How can i find the API? Is it been merged in kernel's master branch
> > > already?
> > >
> > >
> > > Thanks,
> > > zhanghailiang
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > the body of a message to address@hidden
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > .