Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

[Vadim Rozenfeld]

On Fri, 2019-01-11 at 09:50 +0100, Peter Lieven wrote:
> > Am 11.01.2019 um 08:14 schrieb Vadim Rozenfeld <address@hidden
> > >:
> > 
> > > On Thu, 2019-01-10 at 14:57 +0100, Peter Lieven wrote:
> > > > Am 18.12.18 um 15:45 schrieb Peter Lieven:
> > > > > Am 18.12.18 um 14:15 schrieb Vadim Rozenfeld:
> > > > > Peter, I must be missing something here, but what exactly the
> > > > > problem
> > > > > is?
> > > > 
> > > > The issue is that I see concurrent read requests coming in from
> > > > Windows Guest with vioscsi as driver that
> > > > 
> > > > have the same buffer address from guest memory space. I noticed
> > > > this because I have Data Digests enabled
> > > > 
> > > > and the calculated Digest has a wrong CRC32C. This happens
> > > > because
> > > > the CRC is calculated while or after data
> > > > 
> > > > from a second requests reads into the same buffer. I see this
> > > > only
> > > > with Windows and the request size seems
> > > > 
> > > > to be always 4K. This is either a bug in Windows, vioscsi or a
> > > > mad
> > > > application issuing concurrent reads using
> > > > 
> > 
> > vioscsi driver (just like viostor) doesn't allocate any memory by
> > itself, except for some small pieces, needed to build virtio queues
> > and
> > to maintain some internally used data. 
> > 
> > All SCSI Request Block (SRB) memory related jobs are done with
> > StorPortGetScatterGatherList function which just retrieves SG List
> > from
> > the specified SRB. SG List is a set SG Elements, where each
> > elements
> > is
> > 
> > typedef struct _STOR_SCATTER_GATHER_ELEMENT {
> >  STOR_PHYSICAL_ADDRESS PhysicalAddress;
> >  ULONG                 Length;
> >  ULONG_PTR             Reserved;
> > } STOR_SCATTER_GATHER_ELEMENT,
> > *PSTOR_SCATTER_GATHER_ELEMENT;
> > 
> > PhysicalAddress is a physical address of one single page in a data
> > buffer,
> > Length is almost always 4K (page size). Theoretically it can be
> > less
> > for the first chunk if data buffer is not page aligned, or for the
> > last
> > chunk, if buffer size is not rounded up to the page granularity. 
> > 
> > In my understanding, class driver can allocate SG list from non-
> > paged
> > pool, which can explain why you are observing the same memory
> > address
> > for different read/write SRBs, just because class driver can
> > recycle
> > the same pages from the same non-paged pool.
> 
> Vadim, thanks for the explaination. The issue is that I see the same
> buffer address in parallel requests. But if I unterstand you
> correctly this is likely a windows or application issue,
> 

Hi Peter,

Sorry for delay in response.

The entire call stack before it comes to vioscsi's BuildIo routine can
be quite big and complicated. Like this one below, for example.

# ChildEBP RetAddr  Args to Child              
00 aa7235d0 84168cf3 857e7008 875df7e8 878710e0
vioscsi!VioScsiBuildIo+0x3fd (FPO: [Non-Fpo]) (CONV: stdcall)
[x:\work\dev\internal-kvm-guest-drivers-windows\vioscsi\vioscsi.c @
1269]
01 aa723654 84168986 9cf51f08 000000c0 00000000
storport!RaidAdapterPostScatterGatherExecute+0x343 (FPO: [0,25,4])
02 aa723668 8120f418 87871028 00000000 8932d090
storport!RaidpAdapterContinueScatterGather+0x36 (FPO: [Non-Fpo])
03 aa7236b0 84168376 88b0c9d8 87871028 9cf51f08
hal!HalBuildScatterGatherListV2+0x288 (FPO: [Non-Fpo])
04 aa72371c 84167e58 a056c0e8 a053de28 aa723788
storport!RaUnitStartIo+0x256 (FPO: [Non-Fpo])
05 aa723798 84167129 00000000 00000000 a053de28
storport!RaidStartIoPacket+0xcf8 (FPO: [2,23,4])
06 aa7237c0 84166f73 8136c940 a053de28 a056c030
storport!RaidUnitSubmitRequest+0x89 (FPO: [Non-Fpo])
07 aa7237fc 84166e00 8136c940 a056c030 8c65a60f
storport!RaUnitScsiIrp+0x133 (FPO: [Non-Fpo])
08 aa723810 812dd008 a056c030 a053de28 a01e36b8
storport!RaDriverScsiIrp+0x40 (FPO: [Non-Fpo])
09 aa723828 840c46dc 8c6c45e0 00000000 00000000 nt!IofCallDriver+0x48
(FPO: [Non-Fpo])
0a aa723888 840fb438 8fe138c0 a01ef680 6de948f5
CLASSPNP!SubmitTransferPacket+0x1dc (FPO: [Non-Fpo])
0b aa723928 840fb20a 000000c0 0000003f 00000000
CLASSPNP!ClasspModeSense+0xa0 (FPO: [Non-Fpo])
0c aa72393c 840a87c6 8c6c45e0 8fe138c0 000000c0
CLASSPNP!ClassModeSense+0x18 (FPO: [Non-Fpo])
0d aa723978 840a17a3 8c6463c8 8c6c4698 8c6c4698
disk!DiskIoctlGetMediaTypesEx+0x11e (FPO: [Non-Fpo])
0e aa7239a8 840c4ca4 8c6c45e0 8c6463c8 8c6c45e0
disk!DiskDeviceControl+0x223 (FPO: [Non-Fpo])
0f aa723a18 840c37e3 8c6c45e0 8c6463c8 8aac5be4
CLASSPNP!ClassDeviceControlDispatch+0x12c (FPO: [2,21,4])
10 aa723a30 812dd008 8c6c45e0 8c6463c8 8c6463c8
CLASSPNP!ClassGlobalDispatch+0x23 (FPO: [Non-Fpo])
11 aa723a48 84be1bd2 8aac5ae0 8aac5b98 8c646480 nt!IofCallDriver+0x48
(FPO: [Non-Fpo])
12 aa723a68 84be1139 8aac5ae0 8c6463c8 8c6b8f68
partmgr!PmFilterDeviceControl+0x142 (FPO: [Non-Fpo])
13 aa723ab8 812dd008 8aac5ae0 8c6463c8 8c6463c8
partmgr!PmGlobalDispatch+0x39 (FPO: [Non-Fpo])
14 aa723ad4 8155b575 89313c9c 0000030c 031cf8f0 nt!IofCallDriver+0x48
(FPO: [Non-Fpo])
15 aa723bb8 8155afca 00000000 00000000 031cf900
nt!IopXxxControlFile+0x5a5 (FPO: [Non-Fpo])
16 aa723be4 813c6bff 0000030c 00000000 00000000
nt!NtDeviceIoControlFile+0x2a (FPO: [Non-Fpo])
17 aa723be4 77750750 0000030c 00000000 00000000
nt!KiSystemServicePostCall (FPO: [0,3] TrapFrame @ aa723c14)
18 031cf8bc 7774ef6a 746a515e 0000030c 00000000
ntdll!KiFastSystemCallRet (FPO: [0,0,0])
19 031cf8c0 746a515e 0000030c 00000000 00000000
ntdll!NtDeviceIoControlFile+0xa (FPO: [10,0,0])
1a 031cf920 771daa3e 0000030c 002d0c04 00000000
KERNELBASE!DeviceIoControl+0x6e (FPO: [Non-Fpo])
1b 031cf950 5d42bb28 0000030c 002d0c04 00000000
KERNEL32!DeviceIoControlImplementation+0x3e (FPO: [Non-Fpo])
1c 031cf9b8 008fbb96 02cd297c 0000030c 031cfa80
vdsutil!GetDeviceAndMediaType+0xb8 (FPO: [Non-Fpo])
1d 031cfa9c 008f7f0b 031cfb00 efa046c6 00000000
vds!CVdsService::ClaimDiskIfRaw+0xc7 (FPO: [Non-Fpo])
1e 031cfb20 008f7d52 efa046aa 00000003 00000002
vds!CVdsService::AssignDisksToProvidersInClass+0x18e (FPO: [Non-Fpo])
1f 031cfb4c 008f7461 efa04662 008f7300 008f7300
vds!CVdsService::AssignDisksToProviders+0x2f (FPO: [Non-Fpo])
20 031cfb84 771da1a4 00000000 771da180 06226b22
vds!CVdsService::InitializeThread+0x161 (FPO: [Non-Fpo])
21 031cfb98 7772174e 00000000 fbb1c259 00000000
KERNEL32!BaseThreadInitThunk+0x24 (FPO: [Non-Fpo])
22 031cfbe0 77721722 ffffffff 777662f1 00000000
ntdll!__RtlUserThreadStart+0x2b (FPO: [SEH])
23 031cfbf0 00000000 008f7300 00000000 00000000
ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

I can try to help you with investigating the problem, but you have to
guide me how to reproduce it on my development system.


Cheers,
Vadim.

> Best,
> Peter
> 
> > 
> > Best regards,
> > Vadim.
> > 
> > 
> > > > the same buffer. From what I have seen in the dumps of the data
> > > > that has been read it seems not to be one
> > > > 
> > > > common application.
> > > 
> > > 
> > > Has any of the Windows Developers an idea?
> > > 
> > > 
> > > Thanks,
> > > 
> > > Peter
> > > 
> > > 
> > > 
> 
>