[Qemu-devel] Re: [PATCH] megasas: Update to version 1.01

[Nicholas A. Bellinger]

On Tue, 2010-06-08 at 16:15 +0200, Hannes Reinecke wrote:
> This patch updates the megasas HBA emulation to version 1.01.
> It fixes the following issues:
> 
> - Remove hand-crafted inquiry command
> - Remove bounce-buffer for direct commands
> - Implements qdev properties to set 'max_sge', 'max_cmds'.
> - Implement JBOD mode
> - Improve direct command handling
> - Minor cleanups
> 
> Signed-off-by: Hannes Reinecke <address@hidden>
> 

Hi Hannes,

I applied your changes and everything looks good with the exception of
the new MEGASAS_DEFAULT_SGE=80 setting..

> diff --git a/hw/megasas.c b/hw/megasas.c
> index 250c3fb..19569a8 100644
> --- a/hw/megasas.c
> +++ b/hw/megasas.c
> @@ -40,38 +40,17 @@ do { fprintf(stderr, "megasas: error: " fmt , ## 
> __VA_ARGS__);} while (0)
>  #endif
>  
>  /* Static definitions */
> -#define MEGASAS_MAX_FRAMES 1000
> -#define MEGASAS_MAX_SGE 8

<snip>

> +#define MEGASAS_VERSION "1.01"
> +#define MEGASAS_MAX_FRAMES 2048              /* Firmware limit at 65535 */
> +#define MEGASAS_DEFAULT_FRAMES 1000  /* Windows requires this */
> +#define MEGASAS_MAX_SGE 255          /* Firmware limit */
> +#define MEGASAS_DEFAULT_SGE 80

Ok, I have been running some LTP disktest raw bandwith benchmarks with a
256K blocksize with megasas -> TCM_Loop -> TCM/RAMDISK_DR LUNs into a
v2.6.26 x86_64 Linux guest (4 VCPUs and 2048 memory) and I noticed
something interesting..

With the new MEGASAS_DEFAULT_SGE 80 setting for fw_sge, read/write tests
have dropped from the original ~1050 MB/sec to roughly ~400 MB/sec.
Passing in the new qdev option using the old default of max_sge=8 the
speed jumps back up to the range that where previously observed w/o this
patch.  Going a bit further, using max_sge=16 jumps up bandwith up to
~1600 MB/sec, and max_sge=24 takes it up to ~2200 MB/sec..!  Using
max_sge=32 then sharply drops back to ~800 MB/sec, and increasing to
larger values brings bandwith down lower and lower..

Taking a look at the megaraid_sas LLD in the KVM guest, the struct
scsi_host is being registered with sg_tablesize=28 which appears to be
where the sharp dropoff for max_sge > 28 begins to occur.  I see that
MFI_DCMD_CTRL_GET_INFO is returning the configured fw_sge to the guest,
but AFAICT megaraid_sas does not adjust itself to use the larger value
reported by GET_INFO.

So that said, I think we want to use MEGASAS_DEFAULT_SGE 28 to match
what the Linux driver is using.  I have not checked what the equivlient
sg_tablesize for the MSFT LLD is doing, but it appears we need to error
on the conserative side here.  What do you think..?

Best,

--nab