Re: [Qemu-devel] [PATCH v4 00/21] Extended I/O accounting

[Stefan Hajnoczi]

On Wed, Oct 28, 2015 at 05:32:57PM +0200, Alberto Garcia wrote:
> Here's v4 of the series that implements extended I/O accounting for
> block devices.
> 
> Since part of Max's BlockBackend series has already been merged, this
> series can now be applied cleanly on top of the master branch without
> additional dependencies.
> 
> Here's the summary of what this series provides:
> 
>  - New block_acct_failed() and block_acct_invalid() calls.
>    We keep track now of the number of successful, failed and invalid
>    operations (each one separated into read, write and flush). So from
>    the API point of view, BlockDeviceStats contains 6 new fields for
>    those.
> 
>  - idle_time_ns: time since the last I/O operation.
> 
>  - New BlockDeviceTimedStats struct: it has statistics for the I/O
>    during a given interval of time. It keeps minimum, maximum and
>    average latencies for read, write and flush operations.
> 
>    It also keeps the average read and write queue depths.
> 
>  - New 'stats-intervals' option that allows the user to define the
>    intervals used to keep the aforementioned statistics. An arbitrary
>    number of intervals can be specified, the length of each one is in
>    seconds.
> 
>    For the API I opted for a colon-separated list of numbers,
> 
>       stats-intervals=60:3600:86400
> 
>    I also considered something a different syntax,
> 
>       stats-intervals.0.length=60,
>       stats-intervals.1.length=3600,
>       stats-intervals.2.length=86400
> 
>    This one could be useful if we want to specify any other attribute
>    for each interval, but I couldn't come up with any, so I chose the
>    simpler solution.
> 
>  - Two new options, stats-account-invalid and stats-account-failed,
>    which allow the user to decide whether to count invalid and failed
>    operations when computing the idle time and total latency.
> 
> Regards,
> 
> Berto
> 
> v4:
> - Rebase on top of the current master. This series no longer depends
>   on any other.
> - patch 8: clarify that interval_length is in seconds [Stefan]
> - patch 9: rewrite timed_average_sum() so it does not call
>   qemu_clock_get_ns() twice [Stefan]
> 
> v3: https://lists.gnu.org/archive/html/qemu-block/2015-10/msg00785.html
> - Rebased on top of the current master and on Max's BlockBackend
>   series v7
> - patch 4: minor documentation fixes [Stefan]
> - patch 5: s/miliseconds/nanoseconds/ [Stefan]
> - patch 6: dropped, there's no "supports_stats" anymore [Stefan]
> - patch 7 (now 6): explain why block_acct_invalid() does not update
>           total_time_ns[] [Stefan]
> - patch 12 (now 11): don't initialize BlockAcctCookie to { 0 }, it's
>            not needed.
> 
> v2: https://lists.gnu.org/archive/html/qemu-block/2015-10/msg00161.html
> - First complete implementation of the new statistics
> 
> v1: https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg03321.html
> - Initial series containing only the timed average infrastructure.
> 
> Alberto Garcia (21):
>   xen_disk: Account for flush operations
>   ide: Account for write operations correctly
>   block: define 'clock_type' for the accounting code
>   util: Infrastructure for computing recent averages
>   block: Add idle_time_ns to BlockDeviceStats
>   block: Add statistics for failed and invalid I/O operations
>   block: Allow configuring whether to account failed and invalid ops
>   block: Compute minimum, maximum and average I/O latencies
>   block: Add average I/O queue depth to BlockDeviceTimedStats
>   block: New option to define the intervals for collecting I/O
>     statistics
>   qemu-io: Account for failed, invalid and flush operations
>   block: Use QEMU_CLOCK_VIRTUAL for the accounting code in qtest mode
>   iotests: Add test for the block device statistics
>   nvme: Account for failed and invalid operations
>   virtio-blk: Account for failed and invalid operations
>   xen_disk: Account for failed and invalid operations
>   atapi: Account for failed and invalid operations
>   ide: Account for failed and invalid operations
>   macio: Account for failed operations
>   scsi-disk: Account for failed operations
>   block: Update copyright of the accounting code
> 
>  block/accounting.c           | 123 ++++++++++++++-
>  block/block-backend.c        |   1 +
>  block/qapi.c                 |  51 +++++++
>  blockdev.c                   |  53 +++++++
>  hmp.c                        |   4 +-
>  hw/block/nvme.c              |  11 +-
>  hw/block/virtio-blk.c        |   4 +-
>  hw/block/xen_disk.c          |  27 +++-
>  hw/ide/atapi.c               |  31 ++--
>  hw/ide/core.c                |  12 +-
>  hw/ide/macio.c               |  12 +-
>  hw/scsi/scsi-disk.c          |  46 ++++--
>  include/block/accounting.h   |  28 ++++
>  include/qemu/timed-average.h |  64 ++++++++
>  qapi/block-core.json         | 103 ++++++++++++-
>  qemu-io-cmds.c               |   9 ++
>  qmp-commands.hx              |  80 +++++++++-
>  tests/Makefile               |   4 +
>  tests/qemu-iotests/136       | 349 
> +++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/136.out   |   5 +
>  tests/qemu-iotests/group     |   1 +
>  tests/test-timed-average.c   |  90 +++++++++++
>  util/Makefile.objs           |   1 +
>  util/timed-average.c         | 231 ++++++++++++++++++++++++++++
>  24 files changed, 1292 insertions(+), 48 deletions(-)
>  create mode 100644 include/qemu/timed-average.h
>  create mode 100644 tests/qemu-iotests/136
>  create mode 100644 tests/qemu-iotests/136.out
>  create mode 100644 tests/test-timed-average.c
>  create mode 100644 util/timed-average.c
> 
> -- 
> 2.6.1
> 

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

signature.asc
Description: PGP signature