[Qemu-devel] [PATCH v7 00/39] blockdev: BlockBackend and media

[Max Reitz]

This series reworks a lot regarding BlockBackend and media. Basically,
it allows empty BlockBackends, that is BBs without a BDS tree.

Before this series, empty drives are represented by a BlockBackend with
an empty BDS attached to it (a BDS with a NULL driver). However, now we
have BlockBackends, thus an empty drive should be represented by a
BlockBackend without any BDS tree attached to it. This is what this
series does.


Quick and early summary for the v7 changes:
- Rebased
- Addressed Kevin's comments for v6


Justification for each of the patches and their order:

-- Preparation before _is_inserted() patches --
 1: Patch 9 will not take care not to break host floppy support, so that
    support needs to be removed first.
 2: Needed for patch 3, so that blockdev-added BDSs without a BB still
    get the BDRV_O_INCOMING flag set.
 3: Needed for patch 4. Patch 26 is a follow-up after BDS-less BBs are
    allowed.
 4: bdrv_close_all() is broken ("block: Rework bdrv_close_all()"). Patch
    7 will break iotest 071 (actually, just make the problem apparent).
    So this patch is required to work around the issue.
    (with "the issue" being that bdrv_close_all() does not unref() the
    BDSs it is closing, but just force-closes everything, even if the
    BDS may still be in use somewhere)

-- _is_inserted() patches --
 5: General clean-up work, nice to have before patch 7 (and goes in tune
    with patch 6).
 6: Using the same BB as a guest device means that the data read from
    there should be exactly the same. Opening the guest tray should
    therefore result in no data being readable. This is what we then
    need this function for.
 7: General clean-up work (in the _is_inserted() area).
 8: General clean-up work (in the _is_inserted() area).
 9: General clean-up work (also regarding _is_inserted()).
10: Required so inserting a floppy will not result in the tray being
    reported as closed (you need to "push in" the floppy first, using
    blockdev-close-tray). It's here in the "_is_inserted() patches area"
    because I feel like that's a very related topic.

-- Support for BDS-less BBs --
11: Preparation for BDS-less BBs
12: Preparation for BDS-less BBs
13: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
14: Patch 15 removes BlockAcctStats from the BDS, but wr_highest_sector
    is BDS-dependent, so it needs to stay here
15: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
16: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
17: Needed for patch 18
18: Preparation for BDS-less BBs (Removing a BDS tree should retain some
    properties for legacy reasons, which must therefore be stored in the
    BB(RS))
19: Preparation for BDS-less BBs
20: Preparation for BDS-less BBs
21: Preparation for BDS-less BBs
22: Ability to add BDS trees to empty BBs ("inserting a medium")
23: Preparation for BDS-less BBs (needs patch 22)
24: One goal of this series, and fixes the "opening tray" event for
    empty drives when shutting down qemu
25: Needed for patch 26
26: Completion of what patch 3 begun
27: Ability to detach BDS trees from BBs

-- "Atomic" QMP tray operations --
28: blockdev-open-tray
29: blockdev-close-tray
30: blockdev-remove-medium
31: blockdev-insert-medium

-- Reimplementation of change/eject --
32: eject
33: change
34: Clean-up patch

-- New QMP blockdev-change-medium command --
35: New QMP command
36: Use for HMP change command
37: Add flag to that command for changing the read-only access mode
    (which was my original intention for this series)
38: Same flag for HMP

-- Tests --
39: iotests are always nice, so here is one


v7:
- Patch 4: Rebase conflict
- Patch 23: [Kevin]
  - Drop blk_is_available() check in drive_backup_prepare(); the same
    check is done by qmp_drive_backup() already
  - eject_device(): If there is no BDS attached to the BB, return
    immediately without doing anything
  - Simplify qmp_bdrv_open_encrypted()
  - Make last error path in qmp_change_blockdev() more explicit
  - Move the contents of two consecutive if (bs [&& ...]) blocks into a
    single if (bs) block
- Patch 24: Drop the unnecessary and now actually wrong block which sets
  BDRV_O_INCOMING [Kevin]
- Patch 25:
  - Make parameters for extract_common_blockdev_options() optional; we
    won't need the throttle options in bds_tree_init()
  - Reorder the parameters (throttling group and throttle config should
    be together, because that makes sense)
  - Rebase conflict (CONFIG_LINUX_AIO dropped)
- Patch 26: Do not allow throttle configuration for BB-less BDS trees
  [Kevin]
- Patch 33: Rebase conflicts due to changes to patch 23


git-backport-diff against v6:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/39:[----] [--] 'block: Remove host floppy support'
002/39:[----] [--] 'block: Set BDRV_O_INCOMING in bdrv_fill_options()'
003/39:[----] [--] 'blockdev: Allow creation of BDS trees without BB'
004/39:[0002] [FC] 'iotests: Only create BB if necessary'
005/39:[----] [--] 'block: Make bdrv_is_inserted() return a bool'
006/39:[----] [--] 'block: Add blk_is_available()'
007/39:[----] [--] 'block: Make bdrv_is_inserted() recursive'
008/39:[----] [--] 'block/raw_bsd: Drop raw_is_inserted()'
009/39:[----] [--] 'block: Invoke change media CB before NULLing drv'
010/39:[----] [--] 'hw/block/fdc: Implement tray status'
011/39:[----] [--] 'hw/usb-storage: Check whether BB is inserted'
012/39:[----] [--] 'block: Fix BB AIOCB AioContext without BDS'
013/39:[----] [--] 'block: Move guest_block_size into BlockBackend'
014/39:[----] [--] 'block: Remove wr_highest_sector from BlockAcctStats'
015/39:[----] [--] 'block: Move BlockAcctStats into BlockBackend'
016/39:[----] [--] 'block: Move I/O status and error actions into BB'
017/39:[----] [--] 'block/throttle-groups: Make incref/decref public'
018/39:[----] [--] 'block: Add BlockBackendRootState'
019/39:[----] [--] 'block: Make some BB functions fall back to BBRS'
020/39:[----] [--] 'block: Fail requests to empty BlockBackend'
021/39:[----] [--] 'block: Prepare remaining BB functions for NULL BDS'
022/39:[----] [--] 'block: Add blk_insert_bs()'
023/39:[0037] [FC] 'block: Prepare for NULL BDS'
024/39:[0004] [FC] 'blockdev: Do not create BDS for empty drive'
025/39:[0176] [FC] 'blockdev: Pull out blockdev option extraction'
026/39:[0067] [FC] 'blockdev: Allow more options for BB-less BDS tree'
027/39:[----] [--] 'block: Add blk_remove_bs()'
028/39:[----] [--] 'blockdev: Add blockdev-open-tray'
029/39:[----] [--] 'blockdev: Add blockdev-close-tray'
030/39:[----] [--] 'blockdev: Add blockdev-remove-medium'
031/39:[----] [--] 'blockdev: Add blockdev-insert-medium'
032/39:[----] [--] 'blockdev: Implement eject with basic operations'
033/39:[0022] [FC] 'blockdev: Implement change with basic operations'
034/39:[----] [--] 'block: Inquire tray state before tray-moved events'
035/39:[----] [--] 'qmp: Introduce blockdev-change-medium'
036/39:[----] [--] 'hmp: Use blockdev-change-medium for change command'
037/39:[----] [--] 'blockdev: read-only-mode for blockdev-change-medium'
038/39:[----] [--] 'hmp: Add read-only-mode option to change command'
039/39:[----] [--] 'iotests: Add test for change-related QMP commands'


Max Reitz (39):
  block: Remove host floppy support
  block: Set BDRV_O_INCOMING in bdrv_fill_options()
  blockdev: Allow creation of BDS trees without BB
  iotests: Only create BB if necessary
  block: Make bdrv_is_inserted() return a bool
  block: Add blk_is_available()
  block: Make bdrv_is_inserted() recursive
  block/raw_bsd: Drop raw_is_inserted()
  block: Invoke change media CB before NULLing drv
  hw/block/fdc: Implement tray status
  hw/usb-storage: Check whether BB is inserted
  block: Fix BB AIOCB AioContext without BDS
  block: Move guest_block_size into BlockBackend
  block: Remove wr_highest_sector from BlockAcctStats
  block: Move BlockAcctStats into BlockBackend
  block: Move I/O status and error actions into BB
  block/throttle-groups: Make incref/decref public
  block: Add BlockBackendRootState
  block: Make some BB functions fall back to BBRS
  block: Fail requests to empty BlockBackend
  block: Prepare remaining BB functions for NULL BDS
  block: Add blk_insert_bs()
  block: Prepare for NULL BDS
  blockdev: Do not create BDS for empty drive
  blockdev: Pull out blockdev option extraction
  blockdev: Allow more options for BB-less BDS tree
  block: Add blk_remove_bs()
  blockdev: Add blockdev-open-tray
  blockdev: Add blockdev-close-tray
  blockdev: Add blockdev-remove-medium
  blockdev: Add blockdev-insert-medium
  blockdev: Implement eject with basic operations
  blockdev: Implement change with basic operations
  block: Inquire tray state before tray-moved events
  qmp: Introduce blockdev-change-medium
  hmp: Use blockdev-change-medium for change command
  blockdev: read-only-mode for blockdev-change-medium
  hmp: Add read-only-mode option to change command
  iotests: Add test for change-related QMP commands

 block.c                         | 180 ++-------
 block/accounting.c              |   8 -
 block/backup.c                  |  17 +-
 block/block-backend.c           | 389 ++++++++++++++++--
 block/commit.c                  |   3 +-
 block/io.c                      |  10 +-
 block/mirror.c                  |  17 +-
 block/qapi.c                    |  36 +-
 block/raw-posix.c               | 230 +----------
 block/raw_bsd.c                 |   6 -
 block/stream.c                  |   3 +-
 block/throttle-groups.c         |  19 +-
 blockdev.c                      | 855 ++++++++++++++++++++++++++++------------
 blockjob.c                      |   5 +-
 hmp-commands.hx                 |  20 +-
 hmp.c                           |  47 ++-
 hw/block/fdc.c                  |  20 +-
 hw/block/xen_disk.c             |   4 +-
 hw/usb/dev-storage.c            |  30 +-
 include/block/accounting.h      |   3 -
 include/block/block.h           |  16 +-
 include/block/block_int.h       |  25 +-
 include/block/throttle-groups.h |   3 +
 include/qemu/typedefs.h         |   1 +
 include/sysemu/block-backend.h  |  14 +-
 include/sysemu/blockdev.h       |   2 -
 migration/block.c               |   5 +
 monitor.c                       |   4 +
 qapi-schema.json                |   6 +-
 qapi/block-core.json            | 138 ++++++-
 qmp-commands.hx                 | 213 +++++++++-
 qmp.c                           |   9 +-
 tests/fdc-test.c                |   4 +-
 tests/qemu-iotests/071          |  54 ++-
 tests/qemu-iotests/071.out      |  12 +-
 tests/qemu-iotests/081          |  18 +-
 tests/qemu-iotests/081.out      |   5 +-
 tests/qemu-iotests/087          |   2 +-
 tests/qemu-iotests/087.out      |   4 +-
 tests/qemu-iotests/118          | 638 ++++++++++++++++++++++++++++++
 tests/qemu-iotests/118.out      |   5 +
 tests/qemu-iotests/group        |   1 +
 42 files changed, 2248 insertions(+), 833 deletions(-)
 create mode 100755 tests/qemu-iotests/118
 create mode 100644 tests/qemu-iotests/118.out

-- 
2.6.1