[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] GSOC Report: Moving I/O throttling and write notifiers into
From: |
Manos Pitsidianakis |
Subject: |
[Qemu-devel] GSOC Report: Moving I/O throttling and write notifiers into block filter drivers |
Date: |
Mon, 28 Aug 2017 17:57:28 +0300 |
User-agent: |
NeoMutt/20170609-57-1e93be (1.8.3) |
This is a GSOC project summary required for the project's final
submission. As part of GSOC 2017, I took the project of moving two hard
coded block layer features into filter drivers. I/O Throttling is
implemented in block/throttle.c and before write notifiers are split
into a driver for each user of the before write notifier API:
block/backup.c and block/write-threshold.c. Furthermore, work began on
block-insert-node and block-remove-node commands for the QMP interface
to allow runtime insertion and removal of filter drivers. [0]
A lot of thanks to my mentors for their help: Alberto Garcia, Stefan
Hajnoczi and Kevin Wolf.
Terms
=====
The BlockBackend struct (block/block-backend.c) represents the backend part
of a storage device that a VM sees in the QEMU environment. The BlockBackend
has the responsibility to forward I/O requests from the VM down to the
actual underlying storage; a network block device, a qcow2 image etc.
In order to allow for polymorphic storage, a BlockBackend forwards the
requests to an acyclic graph in which the leaves are the terminal I/O
destination, a file or network connection. The BlockDriverState struct
represents a node in this graph, and each node is governed by a specific
driver. Above the leaf-nodes we have format drivers that translate requests
for each format (ie a qcow2 driver, or a raw driver). Backing files are
implemented as chains of nodes that forward read requests to their
children but keep write requests to themselves. This setup allows
different node drivers to intercept requests before they reach their
destination by being inserted into points of interest in the graph. We
call these block filter drivers.
An existing filter driver for example is block/blkverify.c which compares
two children for content parity and reports content mismatches.
I/O Throttling
==============
I/O throttling is done by intercepting I/O requests and throttling them
based on the configured limits (docs/throttle.txt). The interface was
refactored into the throttle driver [1] while the throttling primitives
were left unchanged. The already existing interface of setting limits on
a BlockBackend device is simulated [2] by inserting a hidden to the user
throttle filter node at the root of the BlockBackend with the user's set
limits. Implicitly created filter nodes is not a good solution since
some of the QEMU internals are written without considering filter nodes.
Some patches in the throttle-remove-legacy branch are dedicated to
changing existing behaviour to match the new concept of implicit
filters. In the future management tools should be expected to explicitly
add and remove filter nodes like throttle (except for transient block
job filters which may remain implicit) and there should be no surprises
about the state of the block layer graph for the user.
Throttle groups are categories of drives sharing the same limits with a
round-robin algorithm. Additional effort was spent on making throttle groups
easier to configure by turning them into a separately creatable object (with
-object syntax on command line invocation or object-add in QMP). Their
properties can be set with 'qom-set' commands and retrieved with 'qom-get'.
Write Notifiers
===============
While a backup block job is running, it is important to have knowledge of
writes to the relevant image. Before write notifiers pass the write requests
to the backup job to perform copy on write on the target image with the new
data. Currently this is done on the BlockBackend level. Other block jobs
(commit/mirror) already create implicit nodes in the BDS chain and this
approach was copied and a backup filter driver was created [3], internal to
block/backup.c
The write-threshold feature once enabled via QMP, watches passing write
requests and compares them to a user-given threshold offset. When that
threshold is exceeded an event is delivered to the user and the feature
disables itself. This is used for management tools that need to know when
they have to resize their images. Like backup, this was done in the
BlockBackend level. However it wasn't easy to replace the existing interface
with an implicit filter node like in throttling, so only a separate driver
was created [4] in block/write-threshold.c. Like other filter drivers, it
can be inserted on runtime and removed once it delivers the event and is
spent and should be removed or replaced.
Branches / Patches
==================
The 'throttle' and 'throttle-remove-legacy' patches should be merged soon
after master unfreezes from the 2.10 release. The rest of the patch series
are in final stages of review on qemu-devel except for block-insert-node
which is an RFC [5].
Already merged patches in 2.10
https://github.com/qemu/qemu/commits/v2.10.0-rc4?author=epilys
Already merged patches for 2.11
https://www.mail-archive.com/address@hidden/msg470461.html
[0] [insert-node] block-insert-node and block-remove-node commands
https://github.com/epilys/qemu/commits/insert-node?author=epilys
[1] [throttle] add throttle filter driver
https://github.com/epilys/qemu/commits/throttle?author=epilys
Message-ID: <address@hidden>
https://www.mail-archive.com/address@hidden/msg476047.html
[2] [throttle-remove-legacy] remove legacy throttling interface
https://github.com/epilys/qemu/commits/throttle-remove-legacy?author=epilys
[3] [4] [notify] https://github.com/epilys/qemu/commits/notify?author=epilys
[5] block-insert-node RFC
Message-ID: <address@hidden>
https://www.mail-archive.com/address@hidden/msg473619.html
signature.asc
Description: PGP signature
- [Qemu-devel] GSOC Report: Moving I/O throttling and write notifiers into block filter drivers,
Manos Pitsidianakis <=