[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [POC] colo-proxy in qemu
From: |
Dong, Eddie |
Subject: |
Re: [Qemu-devel] [POC] colo-proxy in qemu |
Date: |
Fri, 24 Jul 2015 02:05:35 +0000 |
BTW, I felt it is better to be called as an agency, rather than a proxy. Any
comments from native speaker?
Thx Eddie
> Hi, all
>
> We are planning to implement colo-proxy in qemu to cache and compare
> packets.
> This module is one of the important component of COLO project and now it is
> still in early stage, so any comments and feedback are warmly welcomed, thanks
> in advance.
>
> ## Background
> COLO FT/HA (COarse-grain LOck-stepping Virtual Machines for Non-stop Service)
> project is a high availability solution. Both Primary VM (PVM) and Secondary
> VM
> (SVM) run in parallel. They receive the same request from client, and generate
> responses in parallel too. If the response packets from PVM and SVM are
> identical, they are released immediately. Otherwise, a VM checkpoint (on
> demand) is conducted.
> Paper:
> http://www.socc2013.org/home/program/a3-dong.pdf?attredirects=0
> COLO on Xen:
> http://wiki.xen.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping
> COLO on Qemu/KVM:
> http://wiki.qemu.org/Features/COLO
>
> By the needs of capturing response packets from PVM and SVM and finding out
> whether they are identical, we introduce a new module to qemu networking
> called colo-proxy.
>
> This document describes the design of the colo-proxy module
>
> ## Glossary
> PVM - Primary VM, which provides services to clients.
> SVM - Secondary VM, a hot standby and replication of PVM.
> PN - Primary Node, the host which PVM runs on
> SN - Secondary Node, the host which SVM runs on
>
> ## Workflow ##
> The following image show the qemu networking packet datapath between
> guest's NIC and qemu's backend in colo-proxy.
>
> +---+ +---+
> |PN | |SN |
> +---+--------------------------+
> +---+--------------------------+ +------------------------------+
> | +-------+ | | +-------+ |
> +--------+ |chkpoint<--------[socket]------->chkpoint +--------+
> |PVM | +---^---+ | | +---+---+ |SVM |
> | | +proxy--v--------+ | | | | |
> | | | | | | | | |
> | +---+ | | +TCP/IP stack+ | | | +-----v-------proxy | +---+ |
> +-|NIC|--+ | | | | | | | | +-|NIC|--+
> | +^-++ | | +--------+ | | | | | +TCP/IP stack-+ | +^--+ |
> | | +------> | | compare| | <-[socket]-forward- | +--------+ | | | |
> | | | | +---+----+ | | | | | | |seq&ack | | <----+ |
> | | | +-----|------+ | | | | | |adjust | | | |
> | | | | | | | | | +--------+ | | |
> | +-----------<+>-----copy&forward-[socket]---> +-------------+ | |
> | +---|---|--------+ | | +------------^----+ |
> | | | | | | |
> | | | | | x |
> | +--+---v----+ | | +-v---------+ |
> | QEMU | backend | | | QEMU | backend | |
> +------------+ (tap) +-----+ +------------+ (tap) +-----+
> +-----------+ +-----------+
>
> ## Our Idea ##
>
> ### Net filter
> In current QEMU, a packet is transported between networking backend(tap)
> and qemu network adapter(NIC) directly. Backend and adapter is linked by
> NetClientState->peer in qemu as following
> +----------------------------------------+
> v |
> +NetClientState+ +------->+NetClientState+ |
> |info->type=TAP| | |info->type=NIC| |
> +--------------+ | +--------------+ |
> | *peer +---+ | *peer +----+
> +--------------+ +--------------+
> |name="tap0" | |name="e1000" |
> +--------------+ +--------------+
> | ... | | ... |
> +--------------+ +--------------+
>
> In COLO QEMU, we insert a net filter named colo-proxy between backend and
> adapter like below:
> typedef struct COLOState {
> NetClientState nc;
> NetClientState *peer;
> } COLOState;
> +------->+NetClientState+ +NetClientState+<--------+
> | |info->type=TAP| |info->type=NIC| |
> | +--------------+ +--------------+ |
> +-----------+ *peer | | *peer +------------+
> | | +--------------+ +--------------+ | |
> | | |name="tap0" | |name="e1000" | | |
> | | +--------------+ +--------------+ | |
> | | | ... | | ... | | |
> | | +--------------+ +--------------+ | |
> | | | |
> | | +-COLOState------------+ +-COLOState------------+ | |
> +--------->+NetClientState+<- - + +---->+NetClientState+<---------+
> | | |info->type=COLO | | | | |info->type=COLO | |
> | | +--------------+ | | | | +--------------+ | |
> +-------+ *peer | | | | | | *peer +-------+
> | +--------------+ | | | | +--------------+ |
> | |name="colo1" | | | | | |name="colo2" | |
> | +--------------+ | | | | +--------------+ |
> | | | | | |
> | +--------------+---------+ | +--------------+ |
> | | *peer | | | | | *peer | |
> | +--------------+ | +---------+--------------+ |
> +----------------------+ +----------------------+
>
> After we insert colo-proxy filter, all packets will pass by this filter and
> more
> important thing is that we can analysis packet by ourselves.
>
> ### QEMU space TCP/IP stack(re-use SLIRP) ### We need a QEMU space TCP/IP
> stack to help us to analysis packet. After looking into QEMU, we found that
> SLIRP
> http://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.
> 29
> is a good choice for us. SLIRP proivdes a full TCP/IP stack within QEMU, it
> can
> help use to handle the packet written to/read from backend(tap) device which
> is
> just like a link layer(L2) packet.
>
> ### packet enqueue and compare ###
> Together with QEMU space TCP/IP stack, we enqueue all packets sent by PVM
> and SVM on Primary QEMU, and then compare the packet payload for each
> connection.
>
> ### Net filter Usage ###
> On both Primary/Secondary host, invoke QEMU with the following parameters
> to insert a net filter(colo-proxy):
> "-netdev tap,id=hn0 -device e1000,netdev=hn0 \
> -netdev colo,id=colo,backend=hn0"
>
- Re: [Qemu-devel] [POC] colo-proxy in qemu, (continued)
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Yang Hongyang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Jason Wang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Yang Hongyang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Jason Wang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Yang Hongyang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Jason Wang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Yang Hongyang, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/27
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Yang Hongyang, 2015/07/27
Re: [Qemu-devel] [POC] colo-proxy in qemu,
Dong, Eddie <=
Re: [Qemu-devel] [POC] colo-proxy in qemu, Jason Wang, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Gonglei, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dong, Eddie, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Jason Wang, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Gonglei, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, zhanghailiang, 2015/07/30
- Re: [Qemu-devel] [POC] colo-proxy in qemu, Dr. David Alan Gilbert, 2015/07/30