RE: [PATCH RFC 00/12] *** mulitple RDMA channels for migration ***

[fengzhimin]

Thanks for your review. I will add more trace_ calls in the next version(V2) 
and modify its according to your suggestions.

-----Original Message-----
From: Dr. David Alan Gilbert [mailto:address@hidden] 
Sent: Thursday, January 16, 2020 3:57 AM
To: fengzhimin <address@hidden>
Cc: address@hidden; address@hidden; address@hidden; address@hidden; 
Zhanghailiang <address@hidden>; address@hidden
Subject: Re: [PATCH RFC 00/12] *** mulitple RDMA channels for migration ***

* Zhimin Feng (address@hidden) wrote:
> From: fengzhimin <address@hidden>
> 
> Currently there is a single channel for RDMA migration, this causes 
> the problem that the network bandwidth is not fully utilized for 
> 25Gigabit NIC. Inspired by the Multifd, we use two RDMA channels to 
> send RAM pages, which we call MultiRDMA.
> 
> We compare the migration performance of MultiRDMA with origin RDMA 
> migration. The VM specifications for migration are as follows:
> - VM use 4k page;
> - the number of VCPU is 4;
> - the total memory is 16Gigabit;
> - use 'mempress' tool to pressurize VM(mempress 8000 500);
> - use 25Gigabit network card to migrate;
> 
> For origin RDMA and MultiRDMA migration, the total migration times of 
> VM are as follows:
> +++++++++++++++++++++++++++++++++++++++++++++++++
> |             | NOT rdma-pin-all | rdma-pin-all |
> +++++++++++++++++++++++++++++++++++++++++++++++++
> | origin RDMA |       18 s       |     23 s     |
> -------------------------------------------------
> |  MultiRDMA  |       13 s       |     18 s     |
> +++++++++++++++++++++++++++++++++++++++++++++++++

Very nice.

> For NOT rdma-pin-all migration, the multiRDMA can improve the total 
> migration time by about 27.8%.
> For rdma-pin-all migration, the multiRDMA can improve the total 
> migration time by about 21.7%.
> 
> Test the multiRDMA migration like this:
> 'virsh migrate --live --rdma-parallel --migrateuri rdma://hostname 
> domain qemu+tcp://hostname/system'

It will take me a while to finish the review; but another general suggestion is 
add more trace_ calls; it will make it easier to diagnose problems later.

Dave

> 
> fengzhimin (12):
>   migration: Add multiRDMA capability support
>   migration: Export the 'migration_incoming_setup' function           
>              and add the 'migrate_use_rdma_pin_all' function
>   migration: Create the multi-rdma-channels parameter
>   migration/rdma: Create multiRDMA migration threads
>   migration/rdma: Create the multiRDMA channels
>   migration/rdma: Transmit initial package
>   migration/rdma: Be sure all channels are created
>   migration/rdma: register memory for multiRDMA channels
>   migration/rdma: Wait for all multiRDMA to complete registration
>   migration/rdma: use multiRDMA to send RAM block for rdma-pin-all mode
>   migration/rdma: use multiRDMA to send RAM block for NOT rdma-pin-all
>                   mode
>   migration/rdma: only register the virt-ram block for MultiRDMA
> 
>  migration/migration.c |   55 +-
>  migration/migration.h |    6 +
>  migration/rdma.c      | 1320 +++++++++++++++++++++++++++++++++++++----
>  monitor/hmp-cmds.c    |    7 +
>  qapi/migration.json   |   27 +-
>  5 files changed, 1285 insertions(+), 130 deletions(-)
> 
> --
> 2.19.1
> 
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK