[Qemu-commits] [qemu/qemu] 939413: colo: modified the payload compare function

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 9394133f8678ea02bbabab11c8e89e576ec8e9e8
      
https://github.com/qemu/qemu/commit/9394133f8678ea02bbabab11c8e89e576ec8e9e8
  Author: Mao Zhongyi <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M net/colo-compare.c

  Log Message:
  -----------
  colo: modified the payload compare function

Modified the function colo_packet_compare_common to prepare for the
tcp packet comparison in the next patch.

Cc: Zhang Chen <address@hidden>
Cc: Li Zhijian <address@hidden>
Cc: Jason Wang <address@hidden>

Signed-off-by: Mao Zhongyi <address@hidden>
Signed-off-by: Li Zhijian <address@hidden>
Signed-off-by: Zhang Chen <address@hidden>
Reviewed-by: Zhang Chen <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: f449c9e549c2f93839a805ea9c7d66c9699794af
      
https://github.com/qemu/qemu/commit/f449c9e549c2f93839a805ea9c7d66c9699794af
  Author: Mao Zhongyi <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M net/colo-compare.c
    M net/colo.c
    M net/colo.h
    M net/trace-events

  Log Message:
  -----------
  colo: compare the packet based on the tcp sequence number

Packet size some time different or when network is busy.
Based on same payload size, but TCP protocol can not
guarantee send the same one packet in the same way,

like that:
We send this payload:
------------------------------
| header |1|2|3|4|5|6|7|8|9|0|
------------------------------

primary:
ppkt1:
----------------
| header |1|2|3|
----------------
ppkt2:
------------------------
| header |4|5|6|7|8|9|0|
------------------------

secondary:
spkt1:
------------------------------
| header |1|2|3|4|5|6|7|8|9|0|
------------------------------

In the original method, ppkt1 and ppkt2 are different in size and
spkt1, so they can't compare and trigger the checkpoint.

I have tested FTP get 200M and 1G file many times, I found that
the performance was less than 1% of the native.

Now I reconstructed the comparison of TCP packets based on the
TCP sequence number. first of all, ppkt1 and spkt1 have the same
starting sequence number, so they can compare, even though their
length is different. And then ppkt1 with a smaller payload length
is used as the comparison length, if the payload is same, send
out the ppkt1 and record the offset(the length of ppkt1 payload)
in spkt1. The next comparison, ppkt2 and spkt1 can be compared
from the recorded position of spkt1.

like that:
----------------
| header |1|2|3| ppkt1
---------|-----|
   |     |
---------v-----v--------------
| header |1|2|3|4|5|6|7|8|9|0| spkt1
---------------|\------------|
         | \offset     |
      ---------v-------------v
      | header |4|5|6|7|8|9|0| ppkt2
      ------------------------

In this way, the performance can reach native 20% in my multiple
tests.

Cc: Zhang Chen <address@hidden>
Cc: Li Zhijian <address@hidden>
Cc: Jason Wang <address@hidden>

Signed-off-by: Mao Zhongyi <address@hidden>
Signed-off-by: Li Zhijian <address@hidden>
Signed-off-by: Zhang Chen <address@hidden>
Reviewed-by: Zhang Chen <address@hidden>
Tested-by: Zhang Chen <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: 18d65d225831995f9163e718ad57e839758314e0
      
https://github.com/qemu/qemu/commit/18d65d225831995f9163e718ad57e839758314e0
  Author: Thomas Huth <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M net/hub.c
    M net/hub.h
    M net/net.c
    M qapi/net.json
    M qemu-options.hx

  Log Message:
  -----------
  net: Allow hubports to connect to other netdevs

QEMU can emulate hubs to connect NICs and netdevs. This is currently
primarily used for the mis-named 'vlan' feature of the networking
subsystem. Now the 'vlan' feature has been marked as deprecated, since
its name is rather confusing and the users often rather mis-configure
their network when trying to use it. But while the 'vlan' parameter
should be removed at one point in time, the basic idea of emulating
a hub in QEMU is still good: It's useful for bundling up the output of
multiple NICs into one single l2tp netdev for example.

Now to be able to use the hubport feature without 'vlan's, there is one
missing piece: The possibility to connect a hubport to a netdev, too.
This patch adds this possibility by introducing a new "netdev=..."
parameter to the hubports.

To bundle up the output of multiple NICs into one socket netdev, you can
now run QEMU with these parameters for example:

qemu-system-ppc64 ... -netdev socket,id=s1,connect=:11122 \
    -netdev hubport,hubid=1,id=h1,netdev=s1 \
    -netdev hubport,hubid=1,id=h2 -device e1000,netdev=h2 \
    -netdev hubport,hubid=1,id=h3 -device virtio-net-pci,netdev=h3

For using the socket netdev, you have got to start another QEMU as the
receiving side first, for example with network dumping enabled:

qemu-system-x86_64 -M isapc -netdev socket,id=s0,listen=:11122 \
    -device ne2k_isa,netdev=s0 \
    -object filter-dump,id=f1,netdev=s0,file=/tmp/dump.dat

After the ppc64 guest tried to boot from both NICs, you can see in the
dump file (using Wireshark, for example), that the output of both NICs
(the e1000 and the virtio-net-pci) has been successfully transfered
via the socket netdev in this case.

Suggested-by: Paolo Bonzini <address@hidden>
Signed-off-by: Thomas Huth <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: 93653066445bfab5f034225892f512af8e465dcd
      
https://github.com/qemu/qemu/commit/93653066445bfab5f034225892f512af8e465dcd
  Author: Thomas Huth <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M hmp-commands.hx
    M net/slirp.c

  Log Message:
  -----------
  net: Allow netdevs to be used with 'hostfwd_add' and 'hostfwd_remove'

It does not make much sense to limit these commands to the legacy 'vlan'
concept only, they should work with the modern netdevs, too. So now
it is possible to use this command with one, two or three parameters.

With one parameter, the command installs a hostfwd rule on the default
"user" network:
    hostfwd_add tcp:...

With two parameters, the command installs a hostfwd rule on a netdev
(that's the new way of using this command):
    hostfwd_add netdev_id tcp:...

With three parameters, the command installs a rule on a 'vlan' (aka hub):
    hostfwd_add hub_id name tcp:...

Same applies to the hostfwd_remove command now.

Signed-off-by: Thomas Huth <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: 74f78b993214c8cf02d8c431dd7eabc6c9f43997
      
https://github.com/qemu/qemu/commit/74f78b993214c8cf02d8c431dd7eabc6c9f43997
  Author: Thomas Huth <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M qemu-options.hx

  Log Message:
  -----------
  qemu-doc: Get rid of "vlan=X" example in the documentation

The vlan concept is marked as deprecated, so we should not use
this for examples in the documentation anymore.

Signed-off-by: Thomas Huth <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: bf4835a4d5338bb7424827715df22570a8adc67c
      
https://github.com/qemu/qemu/commit/bf4835a4d5338bb7424827715df22570a8adc67c
  Author: Philippe Mathieu-Daudé <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M MAINTAINERS

  Log Message:
  -----------
  MAINTAINERS: update Dmitry Fleytman email

gently asked by his automatic reply :)

Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Jason Wang <address@hidden>


  Commit: 0d1442912b3d6591a57ee45fbf28bc91e58d9e76
      
https://github.com/qemu/qemu/commit/0d1442912b3d6591a57ee45fbf28bc91e58d9e76
  Author: Peter Maydell <address@hidden>
  Date:   2018-01-29 (Mon, 29 Jan 2018)

  Changed paths:
    M MAINTAINERS
    M hmp-commands.hx
    M net/colo-compare.c
    M net/colo.c
    M net/colo.h
    M net/hub.c
    M net/hub.h
    M net/net.c
    M net/slirp.c
    M net/trace-events
    M qapi/net.json
    M qemu-options.hx

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into 
staging

# gpg: Signature made Mon 29 Jan 2018 08:14:19 GMT
# gpg:                using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <address@hidden>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
  MAINTAINERS: update Dmitry Fleytman email
  qemu-doc: Get rid of "vlan=X" example in the documentation
  net: Allow netdevs to be used with 'hostfwd_add' and 'hostfwd_remove'
  net: Allow hubports to connect to other netdevs
  colo: compare the packet based on the tcp sequence number
  colo: modified the payload compare function

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/fccfcc6328d1...0d1442912b3d