[Qemu-devel] [PATCH v5 00/38] blockdev: BlockBackend and media

[Max Reitz]

This series reworks a lot regarding BlockBackend and media. Basically,
it allows empty BlockBackends, that is BBs without a BDS tree.

Before this series, empty drives are represented by a BlockBackend with
an empty BDS attached to it (a BDS with a NULL driver). However, now we
have BlockBackends, thus an empty drive should be represented by a
BlockBackend without any BDS tree attached to it. This is what this
series does.


Quick and early summary for the v5 changes:
- Rebase on master
- Addressed Kevin's comments for v4


Justification for each of the patches and their order:

-- Preparation before _is_inserted() patches --
 1: Patch 9 will not take care not to break host floppy support, so that
    support needs to be removed first.
 2: Needed for patch 3, so that blockdev-added BDSs without a BB still
    get the BDRV_O_INCOMING flag set.
 3: Needed for patch 4. Patch 25 is a follow-up after BDS-less BBs are
    allowed.
 4: bdrv_close_all() is broken ("block: Rework bdrv_close_all()"). Patch
    7 will break iotest 071 (actually, just make the problem apparent).
    So this patch is required to work around the issue.
    (with "the issue" being that bdrv_close_all() does not unref() the
    BDSs it is closing, but just force-closes everything, even if the
    BDS may still be in use somewhere)

-- _is_inserted() patches --
 5: General clean-up work, nice to have before patch 7 (and goes in tune
    with patch 6).
 6: Using the same BB as a guest device means that the data read from
    there should be exactly the same. Opening the guest tray should
    therefore result in no data being readable. This is what we then
    need this function for.
 7: General clean-up work (in the _is_inserted() area).
 8: General clean-up work (in the _is_inserted() area).
 9: General clean-up work (also regarding _is_inserted()).
10: Required so inserting a floppy will not result in the tray being
    reported as closed (you need to "push in" the floppy first, using
    blockdev-close-tray). It's here in the "_is_inserted() patches area"
    because I feel like that's a very related topic.

-- Support for BDS-less BBs --
11: Preparation for BDS-less BBs
12: Preparation for BDS-less BBs
13: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
14: Patch 15 removes BlockAcctStats from the BDS, but wr_highest_sector
    is BDS-dependent, so it needs to stay here
15: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
16: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
17: Preparation for BDS-less BBs (Removing a BDS tree should retain some
    properties for legacy reasons, which must therefore be stored in the
    BB(RS))
18: Preparation for BDS-less BBs
19: Preparation for BDS-less BBs
20: Preparation for BDS-less BBs
21: Ability to add BDS trees to empty BBs ("inserting a medium")
22: Preparation for BDS-less BBs (needs patch 21)
23: One goal of this series, and fixes the "opening tray" event for
    empty drives when shutting down qemu
24: Needed for patch 25
25: Completion of what patch 3 begun
26: Ability to detach BDS trees from BBs

-- "Atomic" QMP tray operations --
27: blockdev-open-tray
28: blockdev-close-tray
29: blockdev-remove-medium
30: blockdev-insert-medium

-- Reimplementation of change/eject --
31: eject
32: change
33: Clean-up patch

-- New QMP blockdev-change-medium command --
34: New QMP command
35: Use for HMP change command
36: Add flag to that command for changing the read-only access mode
    (which was my original intention for this series)
37: Same flag for HMP

-- Tests --
38: iotests are always nice, so here is one


v5:
- Patch 1: [Kevin]
  - Drop fd_open() stub, use the FreeBSD implementation for every
    platform
  - s/2\.4/2.5/
- Patch 2: Added, required for patch 3 [Kevin]
- Patch 3: @drv parameter has been dropped from bdrv_open()
- Patch 4: Fix up some blockdev-add invocations [Kevin]
- Patch 7: Good thing we have a generic children list now, we can use
  that for a more generic recursive bdrv_is_inserted() [Kevein]
- Patch 10: As of 2711fd33a4b18c5e35a6f7efe57b5d868def829e (-nodefaults
  for the iotests) the tests previously broken by this patch no longer
  have the affected output, thereby making fixing them unnecessary
- Patch 22: Rebase conflicts due to the @drv dropping patches
- Patch 23:
  - Rebase conflict due the new patch 2 (a good thing)
  - Same as for patch 10: The tests no longer need to be fixed
- Patch 25: Rebase conflicts due to the @drv dropping patches
- Patch 32: Rebase conflicts due to the @drv dropping patches


git-backport-diff against v4:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/38:[0011] [FC] 'block: Remove host floppy support'
002/38:[down] 'block: Set BDRV_O_INCOMING in bdrv_fill_options()'
003/38:[0002] [FC] 'blockdev: Allow creation of BDS trees without BB'
004/38:[0020] [FC] 'iotests: Only create BB if necessary'
005/38:[----] [--] 'block: Make bdrv_is_inserted() return a bool'
006/38:[----] [--] 'block: Add blk_is_available()'
007/38:[0013] [FC] 'block: Make bdrv_is_inserted() recursive'
008/38:[down] 'block/raw_bsd: Drop raw_is_inserted()'
009/38:[----] [--] 'block: Invoke change media CB before NULLing drv'
010/38:[0069] [FC] 'hw/block/fdc: Implement tray status'
011/38:[----] [--] 'hw/usb-storage: Check whether BB is inserted'
012/38:[----] [--] 'block: Fix BB AIOCB AioContext without BDS'
013/38:[----] [--] 'block: Move guest_block_size into BlockBackend'
014/38:[----] [--] 'block: Remove wr_highest_sector from BlockAcctStats'
015/38:[----] [--] 'block: Move BlockAcctStats into BlockBackend'
016/38:[----] [--] 'block: Move I/O status and error actions into BB'
017/38:[----] [--] 'block: Add BlockBackendRootState'
018/38:[----] [--] 'block: Make some BB functions fall back to BBRS'
019/38:[----] [--] 'block: Fail requests to empty BlockBackend'
020/38:[----] [--] 'block: Prepare remaining BB functions for NULL BDS'
021/38:[----] [--] 'block: Add blk_insert_bs()'
022/38:[0010] [FC] 'block: Prepare for NULL BDS'
023/38:[0057] [FC] 'blockdev: Do not create BDS for empty drive'
024/38:[----] [--] 'blockdev: Pull out blockdev option extraction'
025/38:[0004] [FC] 'blockdev: Allow more options for BB-less BDS tree'
026/38:[----] [--] 'block: Add blk_remove_bs()'
027/38:[----] [--] 'blockdev: Add blockdev-open-tray'
028/38:[----] [--] 'blockdev: Add blockdev-close-tray'
029/38:[----] [--] 'blockdev: Add blockdev-remove-medium'
030/38:[----] [--] 'blockdev: Add blockdev-insert-medium'
031/38:[----] [--] 'blockdev: Implement eject with basic operations'
032/38:[0032] [FC] 'blockdev: Implement change with basic operations'
033/38:[----] [--] 'block: Inquire tray state before tray-moved events'
034/38:[----] [-C] 'qmp: Introduce blockdev-change-medium'
035/38:[----] [--] 'hmp: Use blockdev-change-medium for change command'
036/38:[----] [--] 'blockdev: read-only-mode for blockdev-change-medium'
037/38:[----] [--] 'hmp: Add read-only-mode option to change command'
038/38:[----] [--] 'iotests: Add test for change-related QMP commands'


Max Reitz (38):
  block: Remove host floppy support
  block: Set BDRV_O_INCOMING in bdrv_fill_options()
  blockdev: Allow creation of BDS trees without BB
  iotests: Only create BB if necessary
  block: Make bdrv_is_inserted() return a bool
  block: Add blk_is_available()
  block: Make bdrv_is_inserted() recursive
  block/raw_bsd: Drop raw_is_inserted()
  block: Invoke change media CB before NULLing drv
  hw/block/fdc: Implement tray status
  hw/usb-storage: Check whether BB is inserted
  block: Fix BB AIOCB AioContext without BDS
  block: Move guest_block_size into BlockBackend
  block: Remove wr_highest_sector from BlockAcctStats
  block: Move BlockAcctStats into BlockBackend
  block: Move I/O status and error actions into BB
  block: Add BlockBackendRootState
  block: Make some BB functions fall back to BBRS
  block: Fail requests to empty BlockBackend
  block: Prepare remaining BB functions for NULL BDS
  block: Add blk_insert_bs()
  block: Prepare for NULL BDS
  blockdev: Do not create BDS for empty drive
  blockdev: Pull out blockdev option extraction
  blockdev: Allow more options for BB-less BDS tree
  block: Add blk_remove_bs()
  blockdev: Add blockdev-open-tray
  blockdev: Add blockdev-close-tray
  blockdev: Add blockdev-remove-medium
  blockdev: Add blockdev-insert-medium
  blockdev: Implement eject with basic operations
  blockdev: Implement change with basic operations
  block: Inquire tray state before tray-moved events
  qmp: Introduce blockdev-change-medium
  hmp: Use blockdev-change-medium for change command
  blockdev: read-only-mode for blockdev-change-medium
  hmp: Add read-only-mode option to change command
  iotests: Add test for change-related QMP commands

 block.c                        | 180 ++------
 block/accounting.c             |   8 -
 block/backup.c                 |  17 +-
 block/block-backend.c          | 387 +++++++++++++++--
 block/commit.c                 |   3 +-
 block/io.c                     |  10 +-
 block/mirror.c                 |  17 +-
 block/qapi.c                   |  36 +-
 block/raw-posix.c              | 230 +----------
 block/raw_bsd.c                |   6 -
 block/stream.c                 |   3 +-
 blockdev.c                     | 916 +++++++++++++++++++++++++++++------------
 blockjob.c                     |   5 +-
 hmp-commands.hx                |  20 +-
 hmp.c                          |  47 ++-
 hw/block/fdc.c                 |  20 +-
 hw/block/xen_disk.c            |   4 +-
 hw/usb/dev-storage.c           |  30 +-
 include/block/accounting.h     |   3 -
 include/block/block.h          |  16 +-
 include/block/block_int.h      |  25 +-
 include/qemu/typedefs.h        |   1 +
 include/sysemu/block-backend.h |  14 +-
 include/sysemu/blockdev.h      |   2 -
 migration/block.c              |   5 +
 monitor.c                      |   4 +
 qapi-schema.json               |   6 +-
 qapi/block-core.json           | 138 ++++++-
 qmp-commands.hx                | 213 +++++++++-
 qmp.c                          |   9 +-
 tests/fdc-test.c               |   4 +-
 tests/qemu-iotests/071         |  54 ++-
 tests/qemu-iotests/071.out     |  12 +-
 tests/qemu-iotests/081         |  18 +-
 tests/qemu-iotests/081.out     |   5 +-
 tests/qemu-iotests/087         |   2 +-
 tests/qemu-iotests/087.out     |   4 +-
 tests/qemu-iotests/118         | 638 ++++++++++++++++++++++++++++
 tests/qemu-iotests/118.out     |   5 +
 tests/qemu-iotests/group       |   1 +
 40 files changed, 2294 insertions(+), 824 deletions(-)
 create mode 100755 tests/qemu-iotests/118
 create mode 100644 tests/qemu-iotests/118.out

-- 
2.5.2