Re: [Qemu-devel] [PATCH 6/9] vhost-scsi: new device supporting the tcm_vhost Linux kernel module

[Asias He]

[CCing Vadim and Yan]

On Thu, Jul 04, 2013 at 07:00:49AM +0000, Libaiqing wrote:
> Hi asias,
>     1 Window can not boot from vhost-scsi device;
>     2 A non-bootable vhost-scsi device can not be driven by windows guest.(I 
> used the driver virtio-win-0.1-59 version.)
>     
>     Do you have any plan about these?

I can reproduce this issue on win2003 (Linux guest works fine). The guest
hangs in the boot process. OS is on virito-blk disk, vhost-scsi is
attached as the second disk.

Guest probes from target 0 to 255. Only target 1 is available.
vhost-scsi sends VIRTIO_SCSI_S_BAD_TARGET back to target 0 and 255 except
tareget 1. Howerver, from the log we can see, guest is sending to target
0 again and hangs afterwards.

Vadim and Yan, any ideas?

[ 3109.954237] br0: port 2(tap0) entered disabled state
[ 3109.954387] device tap0 left promiscuous mode
[ 3109.954389] br0: port 2(tap0) entered disabled state
[ 3119.534136] device tap0 entered promiscuous mode
[ 3119.534169] br0: port 2(tap0) entered forwarding state
[ 3119.534180] br0: port 2(tap0) entered forwarding state
[ 3120.138334] vhost_scsi_ctl_handle_kick: The handling func for control queue.
[ 3120.138666] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.138669] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51
[ 3120.138671] target=0, tpg=          (null)
[ 3120.138672] send bad target head=0, out=1
[ 3120.138673] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.139219] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.139221] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51
[ 3120.139223] target=1, tpg=ffff88021f8fc800
[ 3120.139225] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 36, 
data_direction: 2
[ 3120.139226] vhost_scsi got command opcode: 0x12, lun: 0
[ 3120.139228] vhost_scsi_map_iov_to_sgl sg ffff8802242bde60 sgl_count 1 is_err 0
[ 3120.139229] Mapping 1 iovecs for 1 pages
[ 3120.139233] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.139261] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3120.139470] vhost_get_vq_desc: head: 0, out: 1 in: 1
[ 3120.139472] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006bd9, 
len: 51
[ 3120.139474] target=1, tpg=ffff88021f8fc800
[ 3120.139475] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 0, 
data_direction: 3
[ 3120.139476] vhost_scsi got command opcode: 0x0, lun: 0
[ 3120.139480] vhost_get_vq_desc: head: 128, out: 1 in: 1
[ 3120.139487] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3120.139554] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.139557] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51
[ 3120.139558] target=1, tpg=ffff88021f8fc800
[ 3120.139560] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 8, 
data_direction: 2
[ 3120.139561] vhost_scsi got command opcode: 0x25, lun: 0
[ 3120.139563] vhost_scsi_map_iov_to_sgl sg ffff8802242bde60 sgl_count 1 is_err 0
[ 3120.139564] Mapping 1 iovecs for 1 pages
[ 3120.139568] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.139575] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3120.139783] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.139785] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51
[ 3120.139787] target=2, tpg=          (null)
[ 3120.139788] send bad target head=0, out=1
[ 3120.139789] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.139938] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.139940] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51

...

[ 3120.170209] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.170304] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.170305] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f28006c15, 
len: 51
[ 3120.170306] target=255, tpg=          (null)
[ 3120.170306] send bad target head=0, out=1
[ 3120.170307] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.474648] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.474652] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f280efe85, 
len: 51
[ 3120.474654] target=1, tpg=ffff88021f8fc800
[ 3120.474656] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 512, 
data_direction: 2
[ 3120.474658] vhost_scsi got command opcode: 0x28, lun: 0
[ 3120.474660] vhost_scsi_map_iov_to_sgl sg ffff880208f70d40 sgl_count 2 is_err 0
[ 3120.474662] Mapping 1 iovecs for 2 pages
[ 3120.474666] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.474680] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3120.532843] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.532845] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f280efe85, 
len: 51
[ 3120.532847] target=1, tpg=ffff88021f8fc800
[ 3120.532848] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 512, 
data_direction: 2
[ 3120.532850] vhost_scsi got command opcode: 0x28, lun: 0
[ 3120.532852] vhost_scsi_map_iov_to_sgl sg ffff8802242bde60 sgl_count 1 is_err 0
[ 3120.532853] Mapping 1 iovecs for 1 pages
[ 3120.532857] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.532863] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3120.532889] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3120.532891] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f280efe85, 
len: 51
[ 3120.532892] target=1, tpg=ffff88021f8fc800
[ 3120.532894] Allocated tv_cmd: ffff880223cbf800 exp_data_len: 512, 
data_direction: 2
[ 3120.532895] vhost_scsi got command opcode: 0x28, lun: 0
[ 3120.532897] vhost_scsi_map_iov_to_sgl sg ffff8802242bde60 sgl_count 1 is_err 0
[ 3120.532898] Mapping 1 iovecs for 1 pages
[ 3120.532901] vhost_get_vq_desc: head: 128, out: 1 in: 2
[ 3120.532906] vhost_scsi_complete_cmd_work tv_cmd ffff880223cbf800 resid 0 
status 0x0
[ 3123.852940] vhost_scsi_ctl_handle_kick: The handling func for control queue.
[ 3123.853694] vhost_get_vq_desc: head: 0, out: 1 in: 2
[ 3123.853696] Calling __copy_from_user: vq->iov[0].iov_base: 00007f8f2e6c53ac, 
len: 51
[ 3123.853697] target=0, tpg=          (null)
[ 3123.853698] send bad target head=0, out=1
[ 3123.853700] vhost_get_vq_desc: head: 128, out: 1 in: 2

> Thanks,
> Baiqing.
> > -----Original Message-----
> > From: Asias He [mailto:address@hidden
> > Sent: Wednesday, July 03, 2013 4:09 PM
> > To: Libaiqing
> > Cc: Paolo Bonzini; Wenchao Xia; address@hidden;
> > address@hidden; Michael S. Tsirkin; Haofeng
> > Subject: Re: [Qemu-devel] [PATCH 6/9] vhost-scsi: new device supporting the
> > tcm_vhost Linux kernel module
> > 
> > On Wed, Jul 03, 2013 at 11:23:26AM +0800, Asias He wrote:
> > > On Fri, Jun 21, 2013 at 10:16:48AM +0000, Libaiqing wrote:
> > > > Hi Asias,
> > > >
> > > > > -----Original Message-----
> > > > > From: Asias He [mailto:address@hidden
> > > > > Sent: Thursday, June 20, 2013 5:39 PM
> > > > > To: Libaiqing
> > > > > Cc: Paolo Bonzini; Wenchao Xia; address@hidden;
> > > > > address@hidden; Michael S. Tsirkin; Haofeng
> > > > > Subject: Re: [Qemu-devel] [PATCH 6/9] vhost-scsi: new device
> > supporting the
> > > > > tcm_vhost Linux kernel module
> > > > >
> > > > > On Thu, Jun 20, 2013 at 08:49:50AM +0000, Libaiqing wrote:
> > > > > > Hi Asias,
> > > > > >     Thanks for your config.
> > > > > >     According to you config,I test booting from vhost device with
> > > > > upstream kernel and qemu,but failed.
> > > > > >
> > > > > >     1 installing guest from cdrom,ok.
> > > > > >     2 booting vhost-scsi,guest fs error occurs.
> > > > > >     3 using fileio backstores,the error is same..
> > > > > >     4 rebooting guest,a log printed:
> > > > > >      (qemu) hw/scsi/virtio-scsi.c:533:virtio_scsi_handle_event:
> > Object
> > > > > 0x7fccae7f2c88 is not an instance of type virtio-scsi-device
> > > > >
> > > > > Paolo, I remember you fixed a similar issue?
> > > > >
> > > > > >     5 using upstream seabios,core dumped.
> > > > > >
> > > > > >     Could you give me some advise to debug this problem ? I can
> > provide
> > > > > more information if need.
> > > > >
> > > > > Can you show me qemu commit id you used? Can you verity that if
> > using the
> > > > > host kernel for guest helps? Does booting directly (without the 
> > > > > install
> > > > > and reboot process) work?
> > > > Qemu commit id is "commit
> > 4eda32f588086b6cd0ec2be6a7a6c131f8c2b427".
> > > >
> > > > Guest kernel updateing is useless.
> > > >
> > > > Booting directly doesn't work too.
> > >
> > >
> > > Hello Libaiqing,
> > >
> > > Sorry for the delay. I will try to reproduce it myself.
> > 
> > I tried 1.5.1 and qemu.git/master and your particular commit
> > 4eda32f588086b6cd0ec2be6a7a6c131f8c2b427.
> > 
> > All work well for me. I used 3.10.0-rc6 as both and guest kernel.
> > 
> > >
> > > >
> > > > > > The qemu cmd:
> > > > > > address@hidden x86_64-softmmu]# ./qemu-system-x86_64
> > -enable-kvm
> > > > > -name fedora   -M pc -m 1024 -smp 2   -drive
> > > > > file=/home/fedora18.iso,if=ide,media=cdrom -device
> > > > > vhost-scsi-pci,wwpn=naa.50014057133e25dc  -monitor stdio   -vga
> > qxl
> > > > > -vnc :1
> > > > > >
> > > > > > The vnc output:
> > > > > > Dracut-initqueue[189]:/dev/mapper/fedora-root:UNEXPECTED
> > > > > INCONSISTENCY;RUN FSCK MANUALLY.
> > > > > > Dracut-initqueue[189]: Warning: e2fsck returned with 4
> > > > > > Dracut-initqueue[189]: Warning: ***An error occurred during the file
> > > > > system check.
> > > > > >
> > > > > > The guest kernel log:
> > > > > > Kernel: virtio-pci 0000:00:04.0: irq 40 for MSI/MSI-X
> > > > > > Kernel: virtio-pci 0000:00:04.0: irq 41 for MSI/MSI-X
> > > > > > Kernel: virtio-pci 0000:00:04.0: irq 42 for MSI/MSI-X
> > > > > > Kernel: virtio-pci 0000:00:04.0: irq 43 for MSI/MSI-X
> > > > > > Kernel: scsi2 : Virtio SCSI HBA
> > > > > > Kernel: scsi 2:0:1:0: Direct-Access LIO-ORG r0
> > > > > > Kernel: sd 2:0:1:0: Attached scsi generic sg1 type 0
> > > > > > Kernel: sd 2:0:1:0: [sda]1258912 512-byte logical .....
> > > > > > Kernel: sd 2:0:1:0: [sda]write protect is off
> > > > > > Kernel: sd 2:0:1:0: [sda]Mode sense :43 00 00 08
> > > > > > Kernel: sd 2:0:1:0: [sda]write cache: disabled, read .....
> > > > > > Kernel: sda sda1 sda2
> > > > > > Kernel: sd 2:0:1:0: [sda] Attached SCSI disk
> > > > > > Dracut-initqueue[189]: Scanning devices sda2 for LVM
> > > > > > Dracut-initqueue[189]: inactive '/dev/fedora/swap'...
> > > > > > Dracut-initqueue[189]: inactive '/dev/fedora/root'...
> > > > > >
> > > > > > The info of host:
> > > > > > address@hidden x86_64-softmmu]# uname -a
> > > > > > Linux fedora121 3.10.0-rc6 #1 SMP Wed Jun 19 19:34:24 CST 2013
> > x86_64
> > > > > x86_64 x86_64 GNU/Linux
> > > > > > address@hidden x86_64-softmmu]# lsmod |grep vhost_scsi
> > > > > > vhost_scsi             49456  5
> > > > > > target_core_mod       282163  14
> > > > >
> > target_core_iblock,target_core_pscsi,iscsi_target_mod,target_core_file,vh
> > > > > ost_scsi
> > > > > > address@hidden x86_64-softmmu]# targetcli
> > > > > > targetcli shell version v2.1.fb26
> > > > > > Copyright 2011 by RisingTide Systems LLC and others.
> > > > > > For help on commands, type 'help'.
> > > > > >
> > > > > > /> ls
> > > > > > o-
> > > > >
> > / 
> > ..............................................................................................................
> > > > > ........... [...]
> > > > > >   o-
> > > > >
> > backstores 
> > ...............................................................................................
> > > > > ............... [...]
> > > > > >   | o-
> > > > >
> > block 
> > ..................................................................................................
> > > > > [Storage Objects: 0]
> > > > > >   | o-
> > > > >
> > fileio 
> > .................................................................................................
> > > > > [Storage Objects: 0]
> > > > > >   | o-
> > > > >
> > pscsi 
> > ..................................................................................................
> > > > > [Storage Objects: 0]
> > > > > >   | o-
> > > > >
> > ramdisk 
> > ................................................................................................
> > > > > [Storage Objects: 1]
> > > > > >   |   o-
> > > > > r0 
> > > > > ...................................................................................................
> > > > > [(6.0GiB) activated]
> > > > > >   o-
> > > > >
> > iscsi 
> > .........................................................................................................
> > > > > ... [Targets: 0]
> > > > > >   o-
> > > > >
> > loopback 
> > ..................................................................................................
> > > > > ....... [Targets: 0]
> > > > > >   o-
> > > > >
> > vhost 
> > .......................................................................................................
> > > > > ..... [Targets: 1]
> > > > > >     o-
> > > > >
> > naa.50014057133e25dc 
> > ............................................................................
> > > > > .................. [TPGs: 1]
> > > > > >       o-
> > > > > tpg1 
> > > > > ...............................................................................................
> > > > > [naa.5001405a70ac3421]
> > > > > >         o-
> > > > >
> > acls 
> > .........................................................................................................
> > .
> > > > > [ACLs: 0]
> > > > > >         o-
> > > > >
> > luns 
> > .........................................................................................................
> > .
> > > > > [LUNs: 1]
> > > > > >           o-
> > > > >
> > lun0 
> > .....................................................................................................
> > > > > [ramdisk/r0]
> > > > > >
> > > > > > Regards,
> > > > > > baiqing
> > > > > > > -----Original Message-----
> > > > > > > From: Asias He [mailto:address@hidden
> > > > > > > Sent: Thursday, June 20, 2013 9:34 AM
> > > > > > > To: Libaiqing
> > > > > > > Cc: Paolo Bonzini; Wenchao Xia; address@hidden;
> > > > > > > address@hidden; Michael S. Tsirkin; Haofeng
> > > > > > > Subject: Re: [Qemu-devel] [PATCH 6/9] vhost-scsi: new device
> > supporting
> > > > > the
> > > > > > > tcm_vhost Linux kernel module
> > > > > > >
> > > > > > > On Wed, Jun 19, 2013 at 12:55:10PM +0000, Libaiqing wrote:
> > > > > > > > Hi paolo,
> > > > > > > >   The vhost-scsi device can be used as boot device?
> > > > > > > >   I tested with your config + 3.10 rc6 + seabios 1.7.2.2,but 
> > > > > > > > failed.
> > > > > > > >   Could you give me some advise to debug this problem ? I can
> > > > > provide
> > > > > > > more information if need.
> > > > > > >
> > > > > > > Boot from vhost-scsi is supposed to work. The seabios you used
> > should
> > > > > be
> > > > > > > fine which contains the fixes for vhost-scsi.
> > > > > > >
> > > > > > > Instead of playing with the /sys/kernel/config/target directly, I
> > really
> > > > > > > recommend using targetcli utils.
> > > > > > >
> > > > > > > Nab, I think we really should write some docs for people to use
> > > > > > > vhost-scsi.
> > > > > > >
> > > > > > > This is how I install and use targetcli in RHEL6. Note you need
> > upstream
> > > > > > > kernel and qemu bits for vhost-scsi.
> > > > > > >
> > > > > > > # yum groupinstall  'Development tools'
> > > > > > > # yum install python-devel epydoc python-simpleparse
> > > > > > >
> > > > > > > # git clone git://github.com/agrover/rtslib-fb.git
> > > > > > > # git clone git://github.com/agrover/targetcli-fb.git
> > > > > > > # git clone git://github.com/agrover/configshell-fb.git
> > > > > > > # for i in rtslib-fb configshell-fb targetcli-fb; do
> > > > > > >           make -C $i rpm
> > > > > > >           yum localinstall $i/dist/*.noarch.rpm
> > > > > > >   done
> > > > > > >
> > > > > > > In targetcli, create a backstore and vhost wwpn, e.g.
> > > > > > > # targetcli
> > > > > > > /> /backstores/ramdisk create r0 1g
> > > > > > > /> /vhost create
> > > > > > > /> cd /vhost/naa.500140527cb6616b/tpg1/luns
> > > > > > > /> create /backstores/ramdisk/r0
> > > > > > >
> > > > > > > # qemu -device vhost-scsi-pci,wwpn=naa.500140527cb6616b ...
> > > > > > >
> > > > > > > Hope this helps.
> > > > > > >
> > > > > > > > Regards,
> > > > > > > > baiqing
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: address@hidden
> > > > > > > > >
> > [mailto:address@hidden
> > > > > On
> > > > > > > > > Behalf Of Paolo Bonzini
> > > > > > > > > Sent: Tuesday, May 28, 2013 4:01 PM
> > > > > > > > > To: Wenchao Xia
> > > > > > > > > Cc: address@hidden; address@hidden;
> > > > > address@hidden;
> > > > > > > > > Michael S. Tsirkin
> > > > > > > > > Subject: Re: [Qemu-devel] [PATCH 6/9] vhost-scsi: new device
> > > > > supporting
> > > > > > > the
> > > > > > > > > tcm_vhost Linux kernel module
> > > > > > > > >
> > > > > > > > > Il 28/05/2013 09:13, Wenchao Xia ha scritto:
> > > > > > > > > >> > From: Nicholas Bellinger <address@hidden>
> > > > > > > > > >> >
> > > > > > > > > >> > The WWPN specified in configfs is passed to "-device
> > > > > > > vhost-scsi-pci".
> > > > > > > > > >> > The tgpt field of the SET_ENDPOINT ioctl is obsolete now,
> > so it is
> > > > > not
> > > > > > > > > >> > available from the QEMU command-line.  Instead, I
> > hardcode it
> > > > > to
> > > > > > > > > zero.
> > > > > > > > > >> >
> > > > > > > > > > Hi, Paolo
> > > > > > > > > >   Any document about how to config it correctly in configfs,
> > > > > before
> > > > > > > > > > invoking qemu with the WWPN number?
> > > > > > > > >
> > > > > > > > > Unfortunately no, but vhost-scsi doesn't have many knobs
> > (unlike
> > > > > > > > > iSCSI for example) so it's quite simple.  Here is an example:
> > > > > > > > >
> > > > > > > > > cd /sys/kernel/config/target
> > > > > > > > > mkdir -p core/fileio_0/fileio
> > > > > > > > > echo
> > > > > > >
> > 'fd_dev_name=/home/pbonzini/test.img,fd_dev_size=5905580032' >
> > > > > > > > > core/fileio_0/fileio/control
> > > > > > > > > echo 1 > core/fileio_0/fileio/enable
> > > > > > > > > mkdir -p vhost/naa.600140554cf3a18e/tpgt_0/lun/lun_0
> > > > > > > > > cd vhost/naa.600140554cf3a18e/tpgt_0
> > > > > > > > > ln -sf ../../../../../core/fileio_0/fileio/
> > lun/lun_0/virtual_scsi_port
> > > > > > > > > echo naa.60014053226f0388 > nexus
> > > > > > > > >
> > > > > > > > > The "nexus" value is the initiator WWN.
> > naa.600140554cf3a18e is
> > > > > the
> > > > > > > > > target WWN that you have to pass to "-device vhost-scsi-pci".
> > > > > > > > >
> > > > > > > > > Paolo
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Asias
> > > > >
> > > > > --
> > > > > Asias
> > >
> > > --
> > > Asias
> > 
> > --
> > Asias

-- 
Asias