[PATCH v2 00/16] Migration: More migration atomic counters

[Juan Quintela]

Hi

In this v2 series:
- More documentation here and there.
- Fix migration_rate_set() to really / XFER_LIMIT_RATIO
- All reviewed patches are in Migration PULL request 20230515
- There are later reviewed patches, but that depend on the first ones
  that are still not reviewed.

Please review.

Thanks, Juan.

Subject: [PULL 00/11] Migration 20230515 patches
Based-on: Message-Id: <20230515123334.58995-1-quintela@redhat.com>

[v1]
In this series:
- play with rate limit
  * document that a value of 0 means no rate-limit
  * change all users of INT64_MAX to use 0
  * Make sure that transferred value is right
    This gets transferred == multifd_bytes + qemu_file_transferred()
    until the completation stage.  Changing all devices is overkill and not 
useful.
  * Move all rate_limit calculations to use atomics instead of 
qemu_file_transferred().
    Use atomics for rate_limit.
  * RDMA
    Adjust counters here and there
    Change the "imaginary" 1 byte transfer to say if it has sent a page or not.
    More cleanups due to this changes
  * multifd: Adjust the number of transferred bytes in the right place and 
right amount
    right place: just after write, now with atomic counters we can
    right ammount: Now that we are in the right place, we can do it right also 
for compressing

Please review.

ToDo: Best described as ToSend:
- qemu_file_transfered() is based on atomics on my branch
- transferred atomic is not needed anymore

ToDo before my next send:

- downtime_bytes, precopy_bytes and postcopy_bytes should be based on
  migration_transfered_bytes and not need a counter of their own.

With that my cleanup would have finishing, moving from:
- total_transferred in QEMUFile (not atomic)
- rate_limit_used in QEMUFile (not atomic)
- multifd_bytes in mig_stats
- transferred in mig_stats (not updated everywhere needed, the
  following ones are based on this one)
- downtime_bytes in mig_stats
- precopy_bytes in mig_stats
- postcopy_bytes in mig_stats

To just:
- qemu_file_transferred in mig_stats
- multifd_bytes in mig_stats
- rdma_bytes in mig_stats

And for each transfer, we only update one of the three, everything
else is derived from this three values.

Later, Juan.

Juan Quintela (16):
  migration: Don't use INT64_MAX for unlimited rate
  migration: Correct transferred bytes value
  migration: Move setup_time to mig_stats
  qemu-file: Account for rate_limit usage on qemu_fflush()
  migration: Move rate_limit_max and rate_limit_used to migration_stats
  migration: Move migration_total_bytes() to migration-stats.c
  migration: Add a trace for migration_transferred_bytes
  migration: Use migration_transferred_bytes() to calculate rate_limit
  migration: We don't need the field rate_limit_used anymore
  migration: Don't abuse qemu_file transferred for RDMA
  migration/RDMA: It is accounting for zero/normal pages in two places
  migration/rdma: Remove QEMUFile parameter when not used
  migration/rdma: Don't use imaginary transfers
  migration: Remove unused qemu_file_credit_transfer()
  migration/rdma: Simplify the function that saves a page
  migration/multifd: Compute transferred bytes correctly

 include/migration/qemu-file-types.h | 12 ++++-
 migration/migration-stats.h         | 73 +++++++++++++++++++++++++++
 migration/migration.h               |  1 -
 migration/options.h                 |  7 ---
 migration/qemu-file.h               | 33 +++----------
 hw/ppc/spapr.c                      |  4 +-
 hw/s390x/s390-stattrib.c            |  2 +-
 migration/block-dirty-bitmap.c      |  2 +-
 migration/block.c                   |  5 +-
 migration/migration-stats.c         | 59 ++++++++++++++++++++++
 migration/migration.c               | 36 ++++++--------
 migration/multifd.c                 | 14 +++---
 migration/options.c                 |  7 ++-
 migration/qemu-file.c               | 77 ++++-------------------------
 migration/ram.c                     | 34 +++++++------
 migration/rdma.c                    | 64 +++++++++++++-----------
 migration/savevm.c                  | 21 ++++++--
 migration/vmstate.c                 |  3 ++
 migration/meson.build               |  2 +-
 migration/trace-events              |  3 ++
 20 files changed, 268 insertions(+), 191 deletions(-)

-- 
2.40.1