[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH v5 0/6] MSG_ZEROCOPY + multifd
From: |
Leonardo Bras |
Subject: |
[PATCH v5 0/6] MSG_ZEROCOPY + multifd |
Date: |
Fri, 12 Nov 2021 02:10:35 -0300 |
This patch series intends to enable MSG_ZEROCOPY in QIOChannel, and make
use of it for multifd migration performance improvement.
Patch #1 creates new callbacks for QIOChannel, allowing the implementation
of zerocopy writing.
Patch #2 reworks qio_channel_socket_writev() so it accepts flags for
that are later passed to sendmsg().
Patch #3 implements writev_zerocopy and flush_zerocopy on QIOChannelSocket,
making use of MSG_ZEROCOPY on Linux.
Patch #4 adds a "zerocopy" migration property, only available with
CONFIG_LINUX, and compiled-out in any other architectures.
This migration property has to be enabled before multifd migration starts.
Patch #5 adds a helper function that allows to see if TLS is going to be used.
This helper will be later used in patch #6.
Patch #6 Makes use of QIOChannelSocket zerocopy implementation on
nocomp multifd migration.
Results:
So far, the resource usage of __sys_sendmsg() reduced 15 times, and the
overall migration took 13-22% less time, based in synthetic cpu workload.
The objective is to reduce migration time in hosts with heavy cpu usage.
---
Changes since v4:
- 3 patches got splitted in 6
- Flush is used for syncing after each iteration, instead of only at the end
- If zerocopy is not available, fail in connect instead of failing on write
- 'multifd-zerocopy' property renamed to 'zerocopy'
- Fail migrations that don't support zerocopy, if it's enabled.
- Instead of checking for zerocopy at each write, save the flags in
MultiFDSendParams->write_flags and use them on write
- Reorganized flag usage in QIOChannelSocket
- A lot of typos fixed
- More doc on buffer restrictions
Changes since v3:
- QIOChannel interface names changed from io_async_{writev,flush} to
io_{writev,flush}_zerocopy
- Instead of falling back in case zerocopy is not implemented, return
error and abort operation.
- Flush now waits as long as needed, or return error in case anything
goes wrong, aborting the operation.
- Zerocopy is now conditional in multifd, being set by parameter
multifd-zerocopy
- Moves zerocopy_flush to multifd_send_sync_main() from multifd_save_cleanup
so migration can abort if flush goes wrong.
- Several other small improvements
Changes since v2:
- Patch #1: One more fallback
- Patch #2: Fall back to sync if fails to lock buffer memory in MSG_ZEROCOPY
send.
Changes since v1:
- Reimplemented the patchset using async_write + async_flush approach.
- Implemented a flush to be able to tell whenever all data was written.
Leonardo Bras (6):
QIOChannel: Add io_writev_zerocopy & io_flush_zerocopy callbacks
QIOChannelSocket: Add flags parameter for writing
QIOChannelSocket: Implement io_writev_zerocopy & io_flush_zerocopy for
CONFIG_LINUX
migration: Add zerocopy parameter for QMP/HMP for Linux
migration: Add migrate_use_tls() helper
multifd: Implement zerocopy write in multifd migration
(multifd-zerocopy)
qapi/migration.json | 18 ++++
include/io/channel-socket.h | 2 +
include/io/channel.h | 94 ++++++++++++++++---
migration/migration.h | 6 ++
migration/multifd.h | 4 +-
io/channel-socket.c | 176 ++++++++++++++++++++++++++++++++++--
io/channel.c | 63 ++++++++++---
migration/channel.c | 6 +-
migration/migration.c | 41 +++++++++
migration/multifd.c | 64 ++++++++++---
migration/ram.c | 29 ++++--
migration/socket.c | 5 +
monitor/hmp-cmds.c | 6 ++
13 files changed, 455 insertions(+), 59 deletions(-)
--
2.33.1