[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] e24154: gluster: Align block-status tail
|
From: |
Peter Maydell |
|
Subject: |
[Qemu-commits] [qemu/qemu] e24154: gluster: Align block-status tail |
|
Date: |
Thu, 16 Sep 2021 03:18:57 -0700 |
Branch: refs/heads/master
Home: https://github.com/qemu/qemu
Commit: e24154d878c11688202628755f8a47e3cba0f52a
https://github.com/qemu/qemu/commit/e24154d878c11688202628755f8a47e3cba0f52a
Author: Max Reitz <mreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/gluster.c
Log Message:
-----------
gluster: Align block-status tail
gluster's block-status implementation is basically a copy of that in
block/file-posix.c, there is only one thing missing, and that is
aligning trailing data extents to the request alignment (as added by
commit 9c3db310ff0).
Note that 9c3db310ff0 mentions that "there seems to be no other block
driver that sets request_alignment and [...]", but while block/gluster.c
does indeed not set request_alignment, block/io.c's
bdrv_refresh_limits() will still default to an alignment of 512 because
block/gluster.c does not provide a byte-aligned read function.
Therefore, unaligned tails can conceivably occur, and so we should apply
the change from 9c3db310ff0 to gluster's block-status implementation.
Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210805143603.59503-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 33ff4c9e081acdded2c90c8c1963f3403e16082b
https://github.com/qemu/qemu/commit/33ff4c9e081acdded2c90c8c1963f3403e16082b
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M include/block/block_int.h
Log Message:
-----------
block: Drop BDS comment regarding bdrv_append()
There is a comment above the BDS definition stating care must be taken
to consider handling newly added fields in bdrv_append().
Actually, this comment should have said "bdrv_swap()" as of 4ddc07cac
(nine years ago), and in any case, bdrv_swap() was dropped in
8e419aefa (six years ago). So no such care is necessary anymore.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210812084148.14458-2-hreitz@redhat.com>
Commit: 0bc329fbb009f8601cec23bf2bc48ead0c5a5fa2
https://github.com/qemu/qemu/commit/0bc329fbb009f8601cec23bf2bc48ead0c5a5fa2
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block.c
M block/io.c
M include/block/block_int.h
Log Message:
-----------
block: block-status cache for data regions
As we have attempted before
(https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg06451.html,
"file-posix: Cache lseek result for data regions";
https://lists.nongnu.org/archive/html/qemu-block/2021-02/msg00934.html,
"file-posix: Cache next hole"), this patch seeks to reduce the number of
SEEK_DATA/HOLE operations the file-posix driver has to perform. The
main difference is that this time it is implemented as part of the
general block layer code.
The problem we face is that on some filesystems or in some
circumstances, SEEK_DATA/HOLE is unreasonably slow. Given the
implementation is outside of qemu, there is little we can do about its
performance.
We have already introduced the want_zero parameter to
bdrv_co_block_status() to reduce the number of SEEK_DATA/HOLE calls
unless we really want zero information; but sometimes we do want that
information, because for files that consist largely of zero areas,
special-casing those areas can give large performance boosts. So the
real problem is with files that consist largely of data, so that
inquiring the block status does not gain us much performance, but where
such an inquiry itself takes a lot of time.
To address this, we want to cache data regions. Most of the time, when
bad performance is reported, it is in places where the image is iterated
over from start to end (qemu-img convert or the mirror job), so a simple
yet effective solution is to cache only the current data region.
(Note that only caching data regions but not zero regions means that
returning false information from the cache is not catastrophic: Treating
zeroes as data is fine. While we try to invalidate the cache on zero
writes and discards, such incongruences may still occur when there are
other processes writing to the image.)
We only use the cache for nodes without children (i.e. protocol nodes),
because that is where the problem is: Drivers that rely on block-status
implementations outside of qemu (e.g. SEEK_DATA/HOLE).
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/307
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210812084148.14458-3-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[hreitz: Added `local_file == bs` assertion, as suggested by Vladimir]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 5a1cfd2150596bddbf429f81097cbd8b861515c0
https://github.com/qemu/qemu/commit/5a1cfd2150596bddbf429f81097cbd8b861515c0
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M include/block/block_int.h
Log Message:
-----------
block: Clarify that @bytes is no limit on *pnum
.bdrv_co_block_status() implementations are free to return a *pnum that
exceeds @bytes, because bdrv_co_block_status() in block/io.c will clamp
*pnum as necessary.
On the other hand, if drivers' implementations return values for *pnum
that are as large as possible, our recently introduced block-status
cache will become more effective.
So, make a note in block_int.h that @bytes is no upper limit for *pnum.
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210812084148.14458-4-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Commit: 869e7ee827ebf9392ed100119cc7e21a81311536
https://github.com/qemu/qemu/commit/869e7ee827ebf9392ed100119cc7e21a81311536
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/file-posix.c
Log Message:
-----------
block/file-posix: Do not force-cap *pnum
bdrv_co_block_status() does it for us, we do not need to do it here.
The advantage of not capping *pnum is that bdrv_co_block_status() can
cache larger data regions than requested by its caller.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210812084148.14458-5-hreitz@redhat.com>
Commit: 72b4cabe5e92b131dcb954691f579e59eb9103e3
https://github.com/qemu/qemu/commit/72b4cabe5e92b131dcb954691f579e59eb9103e3
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/gluster.c
Log Message:
-----------
block/gluster: Do not force-cap *pnum
bdrv_co_block_status() does it for us, we do not need to do it here.
The advantage of not capping *pnum is that bdrv_co_block_status() can
cache larger data regions than requested by its caller.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210812084148.14458-6-hreitz@redhat.com>
Commit: 9dbf6455f4f6bc59e68af033c5813c82d8a1ffff
https://github.com/qemu/qemu/commit/9dbf6455f4f6bc59e68af033c5813c82d8a1ffff
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/iscsi.c
Log Message:
-----------
block/iscsi: Do not force-cap *pnum
bdrv_co_block_status() does it for us, we do not need to do it here.
The advantage of not capping *pnum is that bdrv_co_block_status() can
cache larger data regions than requested by its caller.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210812084148.14458-7-hreitz@redhat.com>
Commit: 81dcb9ca1ff7f5588ed94824f13bc4c5232aca95
https://github.com/qemu/qemu/commit/81dcb9ca1ff7f5588ed94824f13bc4c5232aca95
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/297
M tests/qemu-iotests/iotests.py
Log Message:
-----------
iotests: Fix unspecified-encoding pylint warnings
As of recently, pylint complains when `open()` calls are missing an
`encoding=` specified. Everything we have should be UTF-8 (and in fact,
everything should be UTF-8, period (exceptions apply)), so use that.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210824153540.177128-2-hreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Commit: cc16153f1fdec4bc841f0bcb9317c4639ab42530
https://github.com/qemu/qemu/commit/cc16153f1fdec4bc841f0bcb9317c4639ab42530
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/iotests.py
Log Message:
-----------
iotests: Fix use-{list,dict}-literal warnings
pylint proposes using `[]` instead of `list()` and `{}` instead of
`dict()`, because it is faster. That seems simple enough, so heed its
advice.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210824153540.177128-3-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Commit: 26db7b23ce57f12b8324111a2ed3b2a4ba3db4a4
https://github.com/qemu/qemu/commit/26db7b23ce57f12b8324111a2ed3b2a4ba3db4a4
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/297
Log Message:
-----------
iotests/297: Drop 169 and 199 from the skip list
169 and 199 have been renamed and moved to tests/ (commit a44be0334be:
"iotests: rename and move 169 and 199 tests"), so we can drop them from
the skip list.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210902094017.32902-2-hreitz@redhat.com>
Commit: e2ad17a62d9da45fbcddc3faa3d6ced519a9453a
https://github.com/qemu/qemu/commit/e2ad17a62d9da45fbcddc3faa3d6ced519a9453a
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
Log Message:
-----------
migrate-bitmaps-postcopy-test: Fix pylint warnings
pylint complains that discards1_sha256 and all_discards_sha256 are first
set in non-__init__ methods.
These variables are not really class-variables anyway, so let them
instead be returned by start_postcopy(), thus silencing pylint.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210902094017.32902-3-hreitz@redhat.com>
Commit: d8c2e47dbe998d00dcfe0ca5fc0766a98b69a591
https://github.com/qemu/qemu/commit/d8c2e47dbe998d00dcfe0ca5fc0766a98b69a591
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/tests/migrate-bitmaps-test
Log Message:
-----------
migrate-bitmaps-test: Fix pylint warnings
There are a couple of things pylint takes issue with:
- The "time" import is unused
- The import order (iotests should come last)
- get_bitmap_hash() doesn't use @self and so should be a function
- Semicolons at the end of some lines
- Parentheses after "if"
- Some lines are too long (80 characters instead of 79)
- inject_test_case()'s @name parameter shadows a top-level @name
variable
- "lambda self: mc(self)" were equivalent to just "mc", but in
inject_test_case(), it is not equivalent, so add a comment and disable
the warning locally
- Always put two empty lines after a function
- f'exec: cat > /dev/null' does not need to be an f-string
Fix them.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210902094017.32902-4-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Commit: b90d7a18b615be1e521ee5bf99c816a45099a29b
https://github.com/qemu/qemu/commit/b90d7a18b615be1e521ee5bf99c816a45099a29b
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/tests/mirror-top-perms
Log Message:
-----------
mirror-top-perms: Fix AbnormalShutdown path
The AbnormalShutdown exception class is not in qemu.machine, but in
qemu.machine.machine. (qemu.machine.AbnormalShutdown was enough for
Python to find it in order to run this test, but pylint complains about
it.)
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210902094017.32902-5-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Commit: 098d983ea5ff1e2b504902d6081074f6b3bf36b8
https://github.com/qemu/qemu/commit/098d983ea5ff1e2b504902d6081074f6b3bf36b8
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M tests/qemu-iotests/297
Log Message:
-----------
iotests/297: Cover tests/
297 so far does not check the named tests, which reside in the tests/
directory (i.e. full path tests/qemu-iotests/tests). Fix it.
Thanks to the previous two commits, all named tests pass its scrutiny,
so we do not have to add anything to SKIP_FILES.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210902094017.32902-6-hreitz@redhat.com>
Commit: 66fed30c9cd11854fc878a4eceb507e915d7c9cd
https://github.com/qemu/qemu/commit/66fed30c9cd11854fc878a4eceb507e915d7c9cd
Author: Stefano Garzarella <sgarzare@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/mirror.c
Log Message:
-----------
block/mirror: fix NULL pointer dereference in mirror_wait_on_conflicts()
In mirror_iteration() we call mirror_wait_on_conflicts() with
`self` parameter set to NULL.
Starting from commit d44dae1a7c we dereference `self` pointer in
mirror_wait_on_conflicts() without checks if it is not NULL.
Backtrace:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 mirror_wait_on_conflicts (self=0x0, s=<optimized out>, offset=<optimized
out>, bytes=<optimized out>)
at ../block/mirror.c:172
172 self->waiting_for_op = op;
[Current thread is 1 (Thread 0x7f0908931ec0 (LWP 380249))]
(gdb) bt
#0 mirror_wait_on_conflicts (self=0x0, s=<optimized out>, offset=<optimized
out>, bytes=<optimized out>)
at ../block/mirror.c:172
#1 0x00005610c5d9d631 in mirror_run (job=0x5610c76a2c00, errp=<optimized
out>) at ../block/mirror.c:491
#2 0x00005610c5d58726 in job_co_entry (opaque=0x5610c76a2c00) at ../job.c:917
#3 0x00005610c5f046c6 in coroutine_trampoline (i0=<optimized out>,
i1=<optimized out>)
at ../util/coroutine-ucontext.c:173
#4 0x00007f0909975820 in ?? () at
../sysdeps/unix/sysv/linux/x86_64/__start_context.S:91
from /usr/lib64/libc.so.6
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001404
Fixes: d44dae1a7c ("block/mirror: fix active mirror dead-lock in
mirror_wait_on_conflicts")
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20210910124533.288318-1-sgarzare@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 2f43482733352e4423fcbddadd09265de92ddf58
https://github.com/qemu/qemu/commit/2f43482733352e4423fcbddadd09265de92ddf58
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
A tests/qemu-iotests/tests/migrate-during-backup
A tests/qemu-iotests/tests/migrate-during-backup.out
Log Message:
-----------
tests: add migrate-during-backup
Add a simple test which tries to run migration during backup.
bdrv_inactivate_all() should fail. But due to bug (see next commit with
fix) it doesn't, nodes are inactivated and continued backup crashes
on assertion "assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210911120027.8063-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: a13de40a05478e64726dd9861135d344837f3c30
https://github.com/qemu/qemu/commit/a13de40a05478e64726dd9861135d344837f3c30
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block.c
M tests/qemu-iotests/tests/migrate-during-backup
Log Message:
-----------
block: bdrv_inactivate_recurse(): check for permissions and fix crash
We must not inactivate child when parent has write permissions on
it.
Calling .bdrv_inactivate() doesn't help: actually only qcow2 has this
handler and it is used to flush caches, not for permission
manipulations.
So, let's simply check cumulative parent permissions before
inactivating the node.
This commit fixes a crash when we do migration during backup: prior to
the commit nothing prevents all nodes inactivation at migration finish
and following backup write to the target crashes on assertion
"assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().
After the commit, we rely on the fact that copy-before-write filter
keeps write permission on target node to be able to write to it. So
inactivation fails and migration fails as expected.
Corresponding test now passes, so, enable it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210911120027.8063-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 5b3f7daaec02704b340ebd8ec4011e19098a2b3c
https://github.com/qemu/qemu/commit/5b3f7daaec02704b340ebd8ec4011e19098a2b3c
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
A scripts/simplebench/img_bench_templater.py
A scripts/simplebench/table_templater.py
Log Message:
-----------
simplebench: add img_bench_templater.py
Add simple grammar-parsing template benchmark. New tool consume test
template written in bash with some special grammar injections and
produces multiple tests, run them and finally print a performance
comparison table of different tests produced from one template.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210824101517.59802-2-vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 6d207d3501b3e6c46c71d782df413d3f4cad2dd8
https://github.com/qemu/qemu/commit/6d207d3501b3e6c46c71d782df413d3f4cad2dd8
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-cluster.c
Log Message:
-----------
qcow2: refactor handle_dependencies() loop body
No logic change, just prepare for the following commit. While being
here do also small grammar fix in a comment.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210824101517.59802-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: ff812c55634f767fb0f7f58b942af1a1c44e95cf
https://github.com/qemu/qemu/commit/ff812c55634f767fb0f7f58b942af1a1c44e95cf
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-cluster.c
M tests/qemu-iotests/271
M tests/qemu-iotests/271.out
Log Message:
-----------
qcow2: handle_dependencies(): relax conflict detection
There is no conflict and no dependency if we have parallel writes to
different subclusters of one cluster when the cluster itself is already
allocated. So, relax extra dependency.
Measure performance:
First, prepare build/qemu-img-old and build/qemu-img-new images.
cd scripts/simplebench
./img_bench_templater.py
Paste the following to stdin of running script:
qemu_img=../../build/qemu-img-{old|new}
$qemu_img create -f qcow2 -o extended_l2=on /ssd/x.qcow2 1G
$qemu_img bench -c 100000 -d 8 [-s 2K|-s 2K -o 512|-s $((1024*2+512))] \
-w -t none -n /ssd/x.qcow2
The result:
All results are in seconds
------------------ --------- ---------
old new
-s 2K 6.7 ± 15% 6.2 ± 12%
-7%
-s 2K -o 512 13 ± 3% 11 ± 5%
-16%
-s $((1024*2+512)) 9.5 ± 4% 8.4
-12%
------------------ --------- ---------
So small writes are more independent now and that helps to keep deeper
io queue which improves performance.
271 iotest output becomes racy for three allocation in one cluster.
Second and third writes may finish in different order. Second and
third requests don't depend on each other any more. Still they both
depend on first request anyway. Filter out second and third write
offsets to cover both possible outputs.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210824101517.59802-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
[hreitz: s/ an / and /]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: a1c62436a4f053766892355da0ca35c5df19fc22
https://github.com/qemu/qemu/commit/a1c62436a4f053766892355da0ca35c5df19fc22
Author: Hanna Reitz <hreitz@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M qemu-img.c
Log Message:
-----------
qemu-img: Allow target be aligned to sector size
We cannot write to images opened with O_DIRECT unless we allow them to
be resized so they are aligned to the sector size: Since 9c60a5d1978,
bdrv_node_refresh_perm() ensures that for nodes whose length is not
aligned to the request alignment and where someone has taken a WRITE
permission, the RESIZE permission is taken, too).
Let qemu-img convert pass the BDRV_O_RESIZE flag (which causes
blk_new_open() to take the RESIZE permission) when using cache=none for
the target, so that when writing to it, it can be aligned to the target
sector size.
Without this patch, an error is returned:
$ qemu-img convert -f raw -O raw -t none foo.img /mnt/tmp/foo.img
qemu-img: Could not open '/mnt/tmp/foo.img': Cannot get 'write'
permission without 'resize': Image size is not a multiple of request
alignment
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1994266
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210819101200.64235-1-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Commit: 786c22d9c2c890f7c7fcd2600571a2ef429d65db
https://github.com/qemu/qemu/commit/786c22d9c2c890f7c7fcd2600571a2ef429d65db
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
Log Message:
-----------
qcow2-refcount: improve style of check_refcounts_l2()
- don't use same name for size in bytes and in entries
- use g_autofree for l2_table
- add whitespace
- fix block comment style
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 9a3978a46bc12e0c49b7114983103b07d90cfa1c
https://github.com/qemu/qemu/commit/9a3978a46bc12e0c49b7114983103b07d90cfa1c
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-cluster.c
M block/qcow2.c
M block/qcow2.h
Log Message:
-----------
qcow2: compressed read: simplify cluster descriptor passing
Let's pass the whole L2 entry and not bother with
L2E_COMPRESSED_OFFSET_SIZE_MASK.
It also helps further refactoring that adds generic
qcow2_parse_compressed_l2_entry() helper.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: a6e098462ba6d8585a6f21729b406ab51a70eb03
https://github.com/qemu/qemu/commit/a6e098462ba6d8585a6f21729b406ab51a70eb03
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-cluster.c
M block/qcow2-refcount.c
M block/qcow2.c
M block/qcow2.h
Log Message:
-----------
qcow2: introduce qcow2_parse_compressed_l2_entry() helper
Add helper to parse compressed l2_entry and use it everywhere instead
of open-coding.
Note, that in most places we move to precise coffset/csize instead of
sector-aligned. Still it should work good enough for updating
refcounts.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-4-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: a2debf6506266cc65014a1964c71059dce7b0118
https://github.com/qemu/qemu/commit/a2debf6506266cc65014a1964c71059dce7b0118
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
Log Message:
-----------
qcow2-refcount: introduce fix_l2_entry_by_zero()
Split fix_l2_entry_by_zero() out of check_refcounts_l2() to be
reused in further patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-5-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 5c3216c0460cbedd12994e584f93e0ea63a026ec
https://github.com/qemu/qemu/commit/5c3216c0460cbedd12994e584f93e0ea63a026ec
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
Log Message:
-----------
qcow2-refcount: fix_l2_entry_by_zero(): also zero L2 entry bitmap
We'll reuse the function to fix wrong L2 entry bitmap. Support it now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-6-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 9631c7822ec60eff0701ebf151bd8b9bd5c1d5d4
https://github.com/qemu/qemu/commit/9631c7822ec60eff0701ebf151bd8b9bd5c1d5d4
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
Log Message:
-----------
qcow2-refcount: check_refcounts_l2(): check l2_bitmap
Check subcluster bitmap of the l2 entry for different types of
clusters:
- for compressed it must be zero
- for allocated check consistency of two parts of the bitmap
- for unallocated all subclusters should be unallocated
(or zero-plain)
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Message-Id: <20210914122454.141075-7-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 289ef5f219d1a94b8225c459dc65821b37637a4f
https://github.com/qemu/qemu/commit/289ef5f219d1a94b8225c459dc65821b37637a4f
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
M block/qcow2.h
Log Message:
-----------
qcow2-refcount: check_refcounts_l2(): check reserved bits
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-8-vsementsov@virtuozzo.com>
[hreitz: Separated `type` declaration from statements]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: cd6efd60e9aaa0665ae76c0ec91298eeac2f4c25
https://github.com/qemu/qemu/commit/cd6efd60e9aaa0665ae76c0ec91298eeac2f4c25
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
Log Message:
-----------
qcow2-refcount: improve style of check_refcounts_l1()
- use g_autofree for l1_table
- better name for size in bytes variable
- reduce code blocks nesting
- whitespaces, braces, newlines
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 98bc07d6cd525910c9aec30989fc82f70ddd620c
https://github.com/qemu/qemu/commit/98bc07d6cd525910c9aec30989fc82f70ddd620c
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
M block/qcow2.h
Log Message:
-----------
qcow2-refcount: check_refcounts_l1(): check reserved bits
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-10-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 8fba39515170752c3bcdbb95551d778c94095271
https://github.com/qemu/qemu/commit/8fba39515170752c3bcdbb95551d778c94095271
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block/qcow2-refcount.c
M block/qcow2.h
Log Message:
-----------
qcow2-refcount: check_refblocks(): add separate message for reserved
Split checking for reserved bits out of aligned offset check.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210914122454.141075-11-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 1899bf47375ad40555dcdff12ba49b4b8b82df38
https://github.com/qemu/qemu/commit/1899bf47375ad40555dcdff12ba49b4b8b82df38
Author: Eric Blake <eblake@redhat.com>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M docs/tools/qemu-img.rst
M qemu-img-cmds.hx
M qemu-img.c
M tests/qemu-iotests/122
Log Message:
-----------
qemu-img: Add -F shorthand to convert
Although we have long supported 'qemu-img convert -o
backing_file=foo,backing_fmt=bar', the fact that we have a shortcut -B
for backing_file but none for backing_fmt has made it more likely that
users accidentally run into:
qemu-img: warning: Deprecated use of backing file without explicit backing
format
when using -B instead of -o. For similarity with other qemu-img
commands, such as create and compare, add '-F $fmt' as the shorthand
for '-o backing_fmt=$fmt'. Update iotest 122 for coverage of both
spellings.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210913131735.1948339-1-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Commit: 57b6f58c1d0df757c9311496c32d502925056894
https://github.com/qemu/qemu/commit/57b6f58c1d0df757c9311496c32d502925056894
Author: Peter Maydell <peter.maydell@linaro.org>
Date: 2021-09-15 (Wed, 15 Sep 2021)
Changed paths:
M block.c
M block/file-posix.c
M block/gluster.c
M block/io.c
M block/iscsi.c
M block/mirror.c
M block/qcow2-cluster.c
M block/qcow2-refcount.c
M block/qcow2.c
M block/qcow2.h
M docs/tools/qemu-img.rst
M include/block/block_int.h
M qemu-img-cmds.hx
M qemu-img.c
A scripts/simplebench/img_bench_templater.py
A scripts/simplebench/table_templater.py
M tests/qemu-iotests/122
M tests/qemu-iotests/271
M tests/qemu-iotests/271.out
M tests/qemu-iotests/297
M tests/qemu-iotests/iotests.py
M tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
M tests/qemu-iotests/tests/migrate-bitmaps-test
A tests/qemu-iotests/tests/migrate-during-backup
A tests/qemu-iotests/tests/migrate-during-backup.out
M tests/qemu-iotests/tests/mirror-top-perms
Log Message:
-----------
Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-09-15' into
staging
Block patches:
- Block-status cache for data regions
- qcow2 optimization (when using subclusters)
- iotests delinting, and let 297 (lint checker) cover named iotests
- qcow2 check improvements
- Added -F (target backing file format) option to qemu-img convert
- Mirror job fix
- Fix for when a migration is initiated while a backup job runs
- Fix for uncached qemu-img convert to a volume with 4k sectors (for an
unaligned image)
- Minor gluster driver fix
# gpg: Signature made Wed 15 Sep 2021 18:39:11 BST
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF
* remotes/hreitz/tags/pull-block-2021-09-15: (32 commits)
qemu-img: Add -F shorthand to convert
qcow2-refcount: check_refblocks(): add separate message for reserved
qcow2-refcount: check_refcounts_l1(): check reserved bits
qcow2-refcount: improve style of check_refcounts_l1()
qcow2-refcount: check_refcounts_l2(): check reserved bits
qcow2-refcount: check_refcounts_l2(): check l2_bitmap
qcow2-refcount: fix_l2_entry_by_zero(): also zero L2 entry bitmap
qcow2-refcount: introduce fix_l2_entry_by_zero()
qcow2: introduce qcow2_parse_compressed_l2_entry() helper
qcow2: compressed read: simplify cluster descriptor passing
qcow2-refcount: improve style of check_refcounts_l2()
qemu-img: Allow target be aligned to sector size
qcow2: handle_dependencies(): relax conflict detection
qcow2: refactor handle_dependencies() loop body
simplebench: add img_bench_templater.py
block: bdrv_inactivate_recurse(): check for permissions and fix crash
tests: add migrate-during-backup
block/mirror: fix NULL pointer dereference in mirror_wait_on_conflicts()
iotests/297: Cover tests/
mirror-top-perms: Fix AbnormalShutdown path
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Compare: https://github.com/qemu/qemu/compare/7b7ab2d6c999...57b6f58c1d0d