Re: [PATCH v2 2/2] migration: add support for qatzip compression when doing live migration

[Markus Armbruster]

"you.chen" <you.chen@intel.com> writes:

> Add config and logics to use qatzip for page compression, in order to support 
> qatzip compression better, we collect multipe pages together to do qatzip 
> compression for best performance.
> And we use compile option CONFIG_QATZIP to determine whether should qatzip 
> related code be compiled or not.
>
> Co-developed-by: dennis.wu <dennis.wu@intel.com>
> Signed-off-by: you.chen <you.chen@intel.com>

[...]

> diff --git a/qapi/migration.json b/qapi/migration.json
> index c84fa10e86..6459927c7a 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -644,6 +644,8 @@
>  #                  no compression, 1 means the best compression speed, and 9 
> means best
>  #                  compression ratio which will consume more CPU.
>  #
> +# @compress-with-qat: compress with qat on and off. (Since 8.1)
> +#
>  # @compress-threads: Set compression thread count to be used in live 
> migration,
>  #                    the compression thread count is an integer between 1 
> and 255.
>  #
> @@ -784,7 +786,7 @@
>  { 'enum': 'MigrationParameter',
>    'data': ['announce-initial', 'announce-max',
>             'announce-rounds', 'announce-step',
> -           'compress-level', 'compress-threads', 'decompress-threads',
> +           'compress-level', 'compress-with-qat', 'compress-threads', 
> 'decompress-threads',
>             'compress-wait-thread', 'throttle-trigger-threshold',
>             'cpu-throttle-initial', 'cpu-throttle-increment',
>             'cpu-throttle-tailslow',
> @@ -815,6 +817,8 @@
>  #
>  # @compress-level: compression level
>  #
> +# @compress-with-qat: compression with qat (Since 8.1)
> +#
>  # @compress-threads: compression thread count
>  #
>  # @compress-wait-thread: Controls behavior when all compression threads are
> @@ -954,6 +958,7 @@
>              '*announce-rounds': 'size',
>              '*announce-step': 'size',
>              '*compress-level': 'uint8',
> +            '*compress-with-qat': 'bool',
>              '*compress-threads': 'uint8',
>              '*compress-wait-thread': 'bool',
>              '*decompress-threads': 'uint8',
> @@ -1152,6 +1157,7 @@
>              '*announce-rounds': 'size',
>              '*announce-step': 'size',
>              '*compress-level': 'uint8',
> +            '*compress-with-qat': 'bool',
>              '*compress-threads': 'uint8',
>              '*compress-wait-thread': 'bool',
>              '*decompress-threads': 'uint8',

We already have MigrationCapability compress

    # @compress: Use multiple compression threads to accelerate live migration.
    #            This feature can help to reduce the migration traffic, by 
sending
    #            compressed pages. Please note that if compress and xbzrle are 
both
    #            on, compress only takes effect in the ram bulk stage, after 
that,
    #            it will be disabled and only xbzrle takes effect, this can 
help to
    #            minimize migration traffic. The feature is disabled by default.
    #            (since 2.4 )

and xbzrle

    # @xbzrle: Migration supports xbzrle (Xor Based Zero Run Length Encoding).
    #          This feature allows us to minimize migration traffic for certain 
work
    #          loads, by sending compressed difference of the pages
    #

and MigrationParameters / MigrateSetParameters multifd-compression

    # @multifd-compression: Which compression method to use.
    #                       Defaults to none. (Since 5.0)
    #
    # @multifd-zlib-level: Set the compression level to be used in live
    #                      migration, the compression level is an integer 
between 0
    #                      and 9, where 0 means no compression, 1 means the best
    #                      compression speed, and 9 means best compression 
ratio which
    #                      will consume more CPU.
    #                      Defaults to 1. (Since 5.0)
    #
    # @multifd-zstd-level: Set the compression level to be used in live
    #                      migration, the compression level is an integer 
between 0
    #                      and 20, where 0 means no compression, 1 means the 
best
    #                      compression speed, and 20 means best compression 
ratio which
    #                      will consume more CPU.
    #                      Defaults to 1. (Since 5.0)

where multifd-compression is

    ##
    # @MultiFDCompression:
    #
    # An enumeration of multifd compression methods.
    #
    # @none: no compression.
    # @zlib: use zlib compression method.
    # @zstd: use zstd compression method.
    #
    # Since: 5.0
    ##

How does this all fit together?  It feels like a bunch of featured piled
onto each other, then shaken well.  Or am I confused?

I could use an abstract description of compression in migration.