Re: [Qemu-devel] [PATCH v4 6/7] iotests: Test qemu-img convert --salvage

[Vladimir Sementsov-Ogievskiy]

07.05.2019 23:35, Max Reitz wrote:
> This test converts a simple image to another, but blkdebug injects
> block_status and read faults at some offsets.  The resulting image
> should be the same as the input image, except that sectors that could
> not be read have to be 0.
> 
> Signed-off-by: Max Reitz <address@hidden>
> ---
>   tests/qemu-iotests/251     | 162 +++++++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/251.out |  43 ++++++++++
>   tests/qemu-iotests/group   |   1 +
>   3 files changed, 206 insertions(+)
>   create mode 100755 tests/qemu-iotests/251
>   create mode 100644 tests/qemu-iotests/251.out
> 
> diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
> new file mode 100755
> index 0000000000..508d69769f
> --- /dev/null
> +++ b/tests/qemu-iotests/251
> @@ -0,0 +1,162 @@
> +#!/usr/bin/env bash
> +#
> +# Test qemu-img convert --salvage
> +#
> +# Copyright (C) 2019 Red Hat, Inc.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +#
> +
> +# creator
> address@hidden
> +
> +seq=$(basename $0)
> +echo "QA output created by $seq"
> +
> +status=1     # failure is the default!
> +
> +_cleanup()
> +{
> +    _cleanup_test_img
> +}
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +. ./common.qemu
> +
> +_supported_fmt generic
> +_supported_proto file
> +_supported_os Linux
> +
> +
> +TEST_IMG="$TEST_IMG.orig" _make_test_img 64M
> +
> +$QEMU_IO -c 'write -P 42 0 64M' "$TEST_IMG.orig" | _filter_qemu_io
> +
> +
> +sector_size=512
> +
> +# Offsets on which to fail block-status.  Keep in ascending order so
> +# the indexing done by _filter_offsets will appear in ascending order
> +# in the output as well.
> +status_fail_offsets="$((16 * 1024 * 1024 + 8192))
> +                     $((33 * 1024 * 1024 + 512))"
> +
> +# Offsets on which to fail reads.  Keep in ascending order for the
> +# same reason.
> +# The second element is shared with $status_fail_offsets on purpose.
> +# Starting with the third element, we test what happens when a
> +# continuous range of sectors is inaccessible.
> +read_fail_offsets="$((32 * 1024 * 1024 - 65536))
> +                   $((33 * 1024 * 1024 + 512))
> +                   $(seq $((34 * 1024 * 1024)) $sector_size \
> +                         $((34 * 1024 * 1024 + 4096 - $sector_size)))"
> +
> +
> +# blkdebug must be above the format layer so it can intercept all
> +# block-status events
> +source_img="json:{'driver': 'blkdebug',
> +                  'image': {
> +                      'driver': '$IMGFMT',
> +                      'file': {
> +                          'driver': 'file',
> +                          'filename': '$TEST_IMG.orig'
> +                      }
> +                  },
> +                  'inject-error': ["
> +
> +for ofs in $status_fail_offsets
> +do
> +    source_img+="{ 'event': 'none',
> +                   'iotype': 'block-status',
> +                   'errno': 5,
> +                   'sector': $((ofs / sector_size)) },"
> +done
> +
> +for ofs in $read_fail_offsets
> +do
> +    source_img+="{ 'event': 'none',
> +                   'iotype': 'read',
> +                   'errno': 5,
> +                   'sector': $((ofs / sector_size)) },"
> +done
> +
> +# Remove the trailing comma and terminate @inject-error and json:{}
> +source_img="${source_img%,} ] }"
> +
> +
> +echo
> +
> +
> +_filter_offsets() {
> +    filters=
> +
> +    index=0
> +    for ofs in $2
> +    do
> +        filters+=" -e s/$(printf "$1" $ofs)/status_fail_offset_$index/"

Why not just $ofs ? I've tested, it works for me as well

> +        index=$((index + 1))
> +    done
> +
> +    index=0
> +    for ofs in $3
> +    do
> +        filters+=" -e s/$(printf "$1" $ofs)/read_fail_offset_$index/"

and here.

> +        index=$((index + 1))
> +    done
> +
> +    sed $filters
> +}
> +
> +# While determining the number of allocated sectors in the input
> +# image, we should see one block status warning per element of
> +# $status_fail_offsets.
> +#
> +# Then, the image is read.  Since the block status is queried in
> +# basically the same way, the same warnings as in the previous step
> +# should reappear.  Interleaved with those we should see a read
> +# warning per element of $read_fail_offsets.
> +# Note that $read_fail_offsets and $status_fail_offsets share an
> +# element (read_fail_offset_1 == status_fail_offset_1), so
> +# "status_fail_offset_1" in the output is the same as
> +# "read_fail_offset_1".
> +$QEMU_IMG convert --salvage "$source_img" "$TEST_IMG" 2>&1 \
> +    | _filter_offsets '%i' "$status_fail_offsets" "$read_fail_offsets"
> +
> +echo
> +
> +# The offsets where the block status could not be determined should
> +# have been treated as containing data and thus should be correct in
> +# the output image.
> +# The offsets where reading failed altogether should be 0.  Make them
> +# 0 in the input image, too, so we can compare both images.
> +for ofs in $read_fail_offsets
> +do
> +    $QEMU_IO -c "write -z $ofs $sector_size" "$TEST_IMG.orig" \
> +        | _filter_qemu_io \
> +        | _filter_offsets '%i' '' "$read_fail_offsets"
> +done
> +
> +echo
> +
> +# These should be equal now.
> +$QEMU_IMG compare "$TEST_IMG.orig" "$TEST_IMG"
> +
> +
> +# success, all done
> +echo "*** done"
> +rm -f $seq.full
> +status=0
> diff --git a/tests/qemu-iotests/251.out b/tests/qemu-iotests/251.out
> new file mode 100644
> index 0000000000..75b8796aad
> --- /dev/null
> +++ b/tests/qemu-iotests/251.out
> @@ -0,0 +1,43 @@
> +QA output created by 251
> +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
> +wrote 67108864/67108864 bytes at offset 0
> +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +
> +qemu-img: warning: error while reading block status at offset 
> status_fail_offset_0: Input/output error
> +qemu-img: warning: error while reading block status at offset 
> status_fail_offset_1: Input/output error
> +qemu-img: warning: error while reading block status at offset 
> status_fail_offset_0: Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_0: 
> Input/output error
> +qemu-img: warning: error while reading block status at offset 
> status_fail_offset_1: Input/output error
> +qemu-img: warning: error while reading offset status_fail_offset_1: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_2: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_3: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_4: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_5: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_6: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_7: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_8: 
> Input/output error
> +qemu-img: warning: error while reading offset read_fail_offset_9: 
> Input/output error
> +
> +wrote 512/512 bytes at offset read_fail_offset_0
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_1
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_2
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_3
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_4
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_5
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_6
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_7
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_8
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 512/512 bytes at offset read_fail_offset_9
> +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +
> +Images are identical.
> +*** done
> diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
> index 7ac9a5ea4a..dde113a552 100644
> --- a/tests/qemu-iotests/group
> +++ b/tests/qemu-iotests/group
> @@ -249,3 +249,4 @@
>   247 rw auto quick
>   248 rw auto quick
>   249 rw auto quick
> +251 auto quick

why not rw? (actually, I don't know what means rw group...)


With simplified _filter_offsets (also, if drop $1 parameter, parameters may be 
changed to be $1 and $2)
(or without, if there is a reason, but I don't see it):
Tested-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>

-- 
Best regards,
Vladimir