Re: [Qemu-devel] [POC] colo-proxy in qemu

[Dr. David Alan Gilbert]

* Jason Wang (address@hidden) wrote:
> 
> 
> On 07/27/2015 01:51 PM, Yang Hongyang wrote:
> > On 07/27/2015 12:49 PM, Jason Wang wrote:
> >>
> >>
> >> On 07/27/2015 11:54 AM, Yang Hongyang wrote:
> >>>
> >>>
> >>> On 07/27/2015 11:24 AM, Jason Wang wrote:
> >>>>
> >>>>
> >>>> On 07/24/2015 04:04 PM, Yang Hongyang wrote:
> >>>>> Hi Jason,
> >>>>>
> >>>>> On 07/24/2015 10:12 AM, Jason Wang wrote:
> >>>>>>
> >>>>>>
> >>>>>> On 07/24/2015 10:04 AM, Dong, Eddie wrote:
> >>>>>>> Hi Stefan:
> >>>>>>>       Thanks for your comments!
> >>>>>>>
> >>>>>>>> On Mon, Jul 20, 2015 at 02:42:33PM +0800, Li Zhijian wrote:
> >>>>>>>>> We are planning to implement colo-proxy in qemu to cache and
> >>>>>>>>> compare
> >>>>>>>> packets.
> >>>>>>>>
> >>>>>>>> I thought there is a kernel module to do that?
> >>>>>>>       Yes, that is the previous solution the COLO sub-community
> >>>>>>> choose
> >>>>>>> to go, but we realized it might be not the best choices, and
> >>>>>>> thus we
> >>>>>>> want to bring discussion back here :)  More comments are welcome.
> >>>>>>>
> >>>>>>
> >>>>>> Hi:
> >>>>>>
> >>>>>> Could you pls describe more details on this decision? What's the
> >>>>>> reason
> >>>>>> that you realize it was not the best choice?
> >>>>>
> >>>>> Below is my opinion:
> >>>>>
> >>>>> We realized that there're disadvantages do it in kernel spaces:
> >>>>> 1. We need to recompile kernel: the colo-proxy kernel module is
> >>>>>      implemented as a nf conntrack extension. Adding a extension
> >>>>> need to
> >>>>>      modify the extension struct in-kernel, so recompile kernel is
> >>>>> needed.
> >>>>
> >>>> There's no need to do all in kernel, you can use a separate process to
> >>>> do the comparing and trigger the state sync through monitor.
> >>>
> >>> I don't get it, colo-proxy kernel module using a kthread do the
> >>> comparing and
> >>> trigger the state sync. We implemented it as a nf conntrack extension
> >>> module,
> >>> so we need to extend the extension struct in-kernel, although it just
> >>> needs
> >>> few lines changes to kernel, but a recompile of kernel is needed.
> >>> Are you
> >>> talking about not implement it as a nf conntrack extension?
> >>
> >> Yes, I mean implement the comparing in userspace but not in qemu.
> >
> > Yes, it is an alternative, that requires other components such as
> > netfilter userspace tools, it will add the complexity I think, we
> > wanted to implement a simple solution in QEMU.
> 
> I didn't get the point that why netfilter is needed? Do you mean the
> packet comparing needs to be stateful?

The current kernel world does a few things that take advantage
of the netfilter code:
   1) It's stateful hanging state off conntrack
   2) It modifies sequence numbers off the secondary to match what the
      primary did when it created the stream.
   3) Comparison is on a per-stream basis so that the order of unrelated
      packets doesn't cause a miscompare.

Dave

> > Another reason is
> > that using other userspace tools will affect the performance, the
> > context switch between kernel and userspace may be an overhead.
> 
> We can use 100% time of this process but looks like your RFC of filter
> just did it in iothread?
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK