|
| From: | Gabriel Laupre |
| Subject: | Re: [Qemu-discuss] Error starting domain: vfio-pci |
| Date: | Mon, 8 Jun 2015 15:43:31 -0700 |
Gabriel Laupre <address@hidden> writes:
> Hello,
>
> I tried to use the "macvtap" solution instead of the direct pci
> passthrough. It works. But I soon as I try to attach the pci device in
> direct mode into the VM it doesn't work,
> I am wondering if my versions are too old maybe:
> address@hidden ~]# virsh version
> Compiled against library: libvirt 1.2.8
> Using library: libvirt 1.2.8
> Using API: QEMU 1.2.8
> Running hypervisor: QEMU 2.0.0
>
> I tried to assign the PF0 where my VF are attached to and I have the
> following errors:
Just try attaching the PF. Don't create any VFs on it.
> Unable to complete install: 'internal error: process exited while
> connecting to monitor: 2015-06-05T21:26:23.951795Z qemu-kvm: -device
> vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: error, group 14
> is not viable, please ensure all devices within the iommu_group are bound
> to their vfio bus driver.
This usually means the card has more than one function and they belong
to the same iommu group. If you have both PF0 and PF1 bound to the vfio-stub,
it might complain if you have unbound virtual functions on the device.
Bandan
> 2015-06-05T21:26:23.951831Z qemu-kvm: -device
> vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: failed to get
> group 14
> 2015-06-05T21:26:23.951844Z qemu-kvm: -device
> vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization
> failed.
> 2015-06-05T21:26:23.951857Z qemu-kvm: -device
> vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device 'vfio-pci'
> could not be initialized'
>
> Same error for my PF1.
> Unable to complete install: 'internal error: process exited while
> connecting to monitor: 2015-06-05T21:28:48.689360Z qemu-kvm: -device
> vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: vfio: error, group 14
> is not viable, please ensure all devices within the iommu_group are bound
> to their vfio bus driver.
> 2015-06-05T21:28:48.689414Z qemu-kvm: -device
> vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to get
> group 14
> 2015-06-05T21:28:48.689428Z qemu-kvm: -device
> vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: Device initialization
> failed.
> 2015-06-05T21:28:48.689442Z qemu-kvm: -device
> vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: Device 'vfio-pci'
> could not be initialized
>
> In my host my CPU is an Intel Xeon 5500 i7
> lspci:
> ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
> Architecture Generic Non-Core Registers (rev 05)
> ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
> Architecture System Address Decoder (rev 05)
> ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
> ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0
> (rev 05)
> ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller (rev 05)
> ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Target Address Decoder (rev 05)
> ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Test Registers (rev 05)
> ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 0 Control Registers (rev 05)
> ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 0 Address Registers (rev 05)
> ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 0 Rank Registers (rev 05)
> ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 0 Thermal Control Registers (rev 05)
> ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 1 Control Registers (rev 05)
> ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 1 Address Registers (rev 05)
> ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 1 Rank Registers (rev 05)
> ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 1 Thermal Control Registers (rev 05)
> ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 2 Control Registers (rev 05)
> ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 2 Address Registers (rev 05)
> ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 2 Rank Registers (rev 05)
> ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
> Controller Channel 2 Thermal Control Registers (rev 05)
>
> Using the virt-manager I tried to add the PCI device. I detached it from
> the iommu group using
> virsh nodedev-dettach pci_0000_04_02_4
>
>
> Know the device appears in /dev/vfio/
>
> I changed the right on the file to 666:
> address@hidden ~]# ls -la /dev/vfio/
> crw-rw-rw- 1 root root 246, 1 Jun 5 14:40 24
>
> I tried to boot my VM and I stil get that error:
> Unable to complete install: 'internal error: process exited while
> connecting to monitor: 2015-06-05T21:41:43.855542Z qemu-kvm: -device
> vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
> to open /dev/vfio/vfio: Operation not permitted
> 2015-06-05T21:41:43.855598Z qemu-kvm: -device
> vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
> to setup container for group 24
> 2015-06-05T21:41:43.855613Z qemu-kvm: -device
> vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
> to get group 24
> 2015-06-05T21:41:43.855632Z qemu-kvm: -device
> vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: Device
> initialization failed.
> 2015-06-05T21:41:43.855652Z qemu-kvm: -device
> vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: Device
> 'vfio-pci' could not be initialized'
>
> I am out of ideas :P
>
> Thank you
>
>
> 2015-05-27 18:38 GMT-07:00 Bandan Das <address@hidden>:
>
>>
>> > On May 22, 2015, at 4:50 PM, Gabriel Laupre <address@hidden> wrote:
>> >
>> > I have something new to investigate now :)
>> >
>> Sorry for not following up earlier. Did you investigate further ? Is the
>> PF being used in the host ? Can you try assigning the PF instead ?
>>
>> Also, what processor are you trying this with ?
>>
>> Bandan
>>
>> > 2015-05-22 13:50 GMT-07:00 Gabriel Laupre <address@hidden>:
>> > > You may need to detach all VFs if they are in the same iommu group.
>> > All my VFs are in different iommu group
>> > > Also, check if dmesg has any relevant messages when this error occurs.
>> > I rebooted my machine and the "allow_unsafe_interrupts" was disable.
>> > Therefore I enabled it and I tried to launch again the command. Now I
>> have a different message:
>> >
>> > address@hidden devstack]# /usr/libexec/qemu-kvm -device
>> vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4 -monitor stdio
>> > QEMU 1.5.3 monitor - type 'help' for more information
>> > (qemu) qemu-kvm: -device
>> vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4: vfio: msix_init failed
>> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> vfio: 0000:04:01.4 Error adding PCI capability address@hidden: -22
>> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> Device initialization failed.
>> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> Device 'vfio-pci' could not be initialized
>> >
>> > and in my dmesg I have the new entry:
>> > [14346.555141] vfio-pci 0000:04:01.4: enabling device (0000 -> 0002)
>> >
>> >
>> >
>> > 2015-05-22 13:37 GMT-07:00 Bandan Das <address@hidden>:
>> >
>> > Gabriel Laupre <address@hidden> writes:
>> >
>> > >> Just, try using a different slot number.
>> > > I changed the addr from 0x3 to 0x4 with the existing device under
>> 04.01.4
>> > > (it is a VF from a NIC)/
>> > > I get:
>> > > address@hidden devstack]# /usr/libexec/qemu-kvm -device
>> > > vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4 -monitor stdio
>> > > QEMU 1.5.3 monitor - type 'help' for more information
>> > > (qemu) qemu-kvm: -device
>> > > vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4: vfio: failed to
>> set
>> > > iommu for container: Operation not permitted
>> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> > > vfio: failed to setup container for group 22
>> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> > > vfio: failed to get group 22
>> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> > > Device initialization failed.
>> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
>> > > Device 'vfio-pci' could not be initialized
>> > >
>> > > under the group 22 I have the said device
>> > > # /sys/kernel/iommu_groups/22/devices/0000:04:01.4
>> > >
>> > > I dettached it before using the command
>> > > # virsh nodedev-detach pci_0000_04_01_4
>> >
>> > You may need to detach all VFs if they are in the same iommu group.
>> >
>> > Also, check if dmesg has any relevant messages when this error
>> > occurs.
>> >
>> > > My goodness, what's going on :P Thank for you time Bandan, I really
>> > > appreciate
>> > >
>> > > 2015-05-22 13:14 GMT-07:00 Bandan Das <address@hidden>:
>> > >
>> > >> Gabriel Laupre <address@hidden> writes:
>> > >>
>> > >> > Actually, I get the same error if I enter a PCI address completely
>> random
>> > >> > (for example 04.01.5, which doesn't exist in my system)
>> > >>
>> > >> No, that has nothing to do with the pci address of the host device
>> you are
>> > >> trying to assign. It's telling you that that slot is already used up
>> in
>> > >> the guest.
>> > >>
>> > >> Just, try using a different slot number.
>> > >>
>> > >> > 2015-05-22 12:00 GMT-07:00 Bandan Das <address@hidden>:
>> > >> >
>> > >> >> Gabriel Laupre <address@hidden> writes:
>> > >> >>
>> > >> >> >>To rule out if it's a problem with your guest xml. Let me know
>> if you
>> > >> are
>> > >> >> > interested and I will write down the steps.
>> > >> >> >
>> > >> >> > I am way more than interested to try that :) Thanks a lot
>> > >> >>
>> > >> >> Ok, Based on your command line below, I am assuming that you have a
>> > >> >> multifunction card. You might need to bind all functions to the
>> > >> >> stub driver if they are in the same iommu group or else vfio will
>> > >> complain.
>> > >> >>
>> > >> >> virsh nodedev-list | grep 04_10
>> > >> >>
>> > >> >> The output would be something like
>> > >> >> pci_0000_04_10_0
>> > >> >> pci_0000_04_10_1 and so on.
>> > >> >>
>> > >> >> For each, assuming they are the same card, you have to bind them
>> to the
>> > >> >> stub driver:
>> > >> >> virsh nodedev-detach pci_0000_04_10_0
>> > >> >> virsh nodedev-detach pci_0000_04_10_1 ...
>> > >> >>
>> > >> >> Shut down all guests and make sure no qemu processes are running
>> > >> >> and then run qemu:
>> > >> >> /usr/libexec/qemu-kvm -device
>> > >> >> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3 -monitor stdio
>> > >> >>
>> > >> >> I think this should be enough to spit out the error, there's no
>> need to
>> > >> >> actually
>> > >> >> boot a guest image.
>> > >> >>
>> > >> >> > 2015-05-22 10:46 GMT-07:00 Bandan Das <address@hidden>:
>> > >> >> >
>> > >> >> >> Gabriel Laupre <address@hidden> writes:
>> > >> >> >>
>> > >> >> >> >> Thanks for the update. Did you get a chance to run qemu
>> standalone
>> > >> >> >> > without libvirt?
>> > >> >> >> > How can I do that, what are the purpose to do so and why to
>> try
>> > >> that?
>> > >> >> >> > (thank you ^^)
>> > >> >> >> To rule out if it's a problem with your guest xml. Let me know
>> if you
>> > >> >> are
>> > >> >> >> interested and I will write down the steps.
>> > >> >> >>
>> > >> >> >> > 2015-05-21 13:32 GMT-07:00 Bandan Das <address@hidden>:
>> > >> >> >> >
>> > >> >> >> >> Gabriel Laupre <address@hidden> writes:
>> > >> >> >> >>
>> > >> >> >> >> >> AppArmor is Debian based, but that doesn't stop you from
>> > >> >> >> >> > running/installing it on other distros
>> > >> >> >> >> >
>> > >> >> >> >> >> $ dmesg | grep apparmor
>> > >> >> >> >> >
>> > >> >> >> >> > The service isn't started nor present in my machine. Thank
>> you
>> > >> for
>> > >> >> the
>> > >> >> >> >> hint
>> > >> >> >> >> > though.
>> > >> >> >> >> > I am still stuck with this error and it is not dependent
>> of my
>> > >> >> Intel
>> > >> >> >> >> NIC, I
>> > >> >> >> >> > used another one and I still got the same error message.
>> > >> >> >> >> > ps: tks to remind me to send to the mailing list:)
>> > >> >> >> >>
>> > >> >> >> >> Thanks for the update. Did you get a chance to run qemu
>> standalone
>> > >> >> >> without
>> > >> >> >> >> libvirt ?
>> > >> >> >> >>
>> > >> >> >> >> > 2015-05-21 12:32 GMT-07:00 Karl Apsite <
>> > >> >> address@hidden>:
>> > >> >> >> >> >
>> > >> >> >> >> >> AppArmor is Debian based, but that doesn't stop you from
>> > >> >> >> >> >> running/installing it
>> > >> >> >> >> >> on other distros
>> > >> >> >> >> >>
>> > >> >> >> >> >> $ dmesg | grep apparmor
>> > >> >> >> >> >>
>> > >> >> >> >> >> On 05/20/2015 02:56 PM, Bandan Das wrote:
>> > >> >> >> >> >> > Gabriel Laupre <address@hidden> writes:
>> > >> >> >> >> >> >
>> > >> >> >> >> >> >> Mmmhh,
>> > >> >> >> >> >> >> My SELinux is disabled. Is Apparmor not only on
>> > >> debian/ubuntu
>> > >> >> and
>> > >> >> >> >> suse,
>> > >> >> >> >> >> am
>> > >> >> >> >> >> >> I wrong? I have no idea on that :)
>> > >> >> >> >> >> >
>> > >> >> >> >> >> > Yeah, me neither :) I am just trying to rule out all
>> > >> >> possibilities.
>> > >> >> >> >> >> >
>> > >> >> >> >> >> >> 2015-05-20 11:23 GMT-07:00 Bandan Das <address@hidden
>> >:
>> > >> >> >> >> >> >>
>> > >> >> >> >> >> >>> Oh and one more thing! You already answered before
>> but just
>> > >> >> >> wanted
>> > >> >> >> >> to
>> > >> >> >> >> >> >>> confirm
>> > >> >> >> >> >> >>> that you don't have apparmor running, right ?
>> > >> >> >> >> >> >>>
>> > >> >> >> >> >> >>> Bandan Das <address@hidden> writes:
>> > >> >> >> >> >> >>>
>> > >> >> >> >> >> >>>> Gabriel Laupre <address@hidden> writes:
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>>>> Yes, indeed it is. What distro is this ? Do you
>> have
>> > >> >> SELinux
>> > >> >> >> or
>> > >> >> >> >> any
>> > >> >> >> >> >> >>> other
>> > >> >> >> >> >> >>>>> security feature enabled ? Can you please verify
>> that the
>> > >> >> file
>> > >> >> >> >> has a
>> > >> >> >> >> >> >>>>> appropriate label if SELinux is enabled ? (ls -lZ
>> > >> >> >> /dev/vfio/vfio)
>> > >> >> >> >> >> >>>>> My distrib:
>> > >> >> >> >> >> >>>>> address@hidden ~]# cat /proc/version
>> > >> >> >> >> >> >>>>> Linux version 3.10.0-229.1.2.el7.x86_64 (
>> > >> >> >> >> >> >>> address@hidden)
>> > >> >> >> >> >> >>>>> (gcc version 4.8.2 20140120 (Red Hat 4.8.2-16)
>> (GCC) ) #1
>> > >> >> SMP
>> > >> >> >> Fri
>> > >> >> >> >> >> Mar 27
>> > >> >> >> >> >> >>>>> 03:04:26 UTC 2015
>> > >> >> >> >> >> >>>>> address@hidden ~]# cat /etc/centos-release
>> > >> >> >> >> >> >>>>> CentOS Linux release 7.1.1503 (Core)
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>> address@hidden ~]# ls -lZ /dev/vfio/vfio
>> > >> >> >> >> >> >>>>> crw-rw-rw- root root ?
>> > >> >> >> >> /dev/vfio/vfio
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>> SELinux is disabled:
>> > >> >> >> >> >> >>>>> address@hidden ~]# getenforce
>> > >> >> >> >> >> >>>>> Disabled
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>> I guess no other security feature is enabled that I
>> am
>> > >> aware
>> > >> >> >> of. I
>> > >> >> >> >> >> once
>> > >> >> >> >> >> >>> had
>> > >> >> >> >> >> >>>>> a message saying that it can be one of the following
>> > >> issues
>> > >> >> >> >> (listing
>> > >> >> >> >> >> the
>> > >> >> >> >> >> >>>>> 5). So I guess it can be any combination of those
>> issues,
>> > >> >> even
>> > >> >> >> >> >> something
>> > >> >> >> >> >> >>>>> completely different.
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>> Ugh, I am out of options! Can you please try a few
>> more
>> > >> >> things:
>> > >> >> >> Can
>> > >> >> >> >> >> you
>> > >> >> >> >> >> >>> try
>> > >> >> >> >> >> >>>> running qemu directly and see if you see the same
>> > >> behavior ?
>> > >> >> If
>> > >> >> >> you
>> > >> >> >> >> >> still
>> > >> >> >> >> >> >>>> haven't tried running as root, please try that too.
>> Also,
>> > >> >> please
>> > >> >> >> >> check
>> > >> >> >> >> >> >>> dmesg
>> > >> >> >> >> >> >>>> for any vfio related errors.
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>>> libvirtError: internal error: process exited while
>> > >> >> connecting
>> > >> >> >> to
>> > >> >> >> >> >> >>>>> monitor: 2015-05-19T21:46:21.935043Z qemu-kvm:
>> -device
>> > >> >> >> >> >> >>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed to
>> > >> >> >> >> >> >>>>> open /dev/vfio/vfio: Operation not permitted
>> > >> >> >> >> >> >>>> Well, this is the first error from
>> > >> vfio_connect_container()
>> > >> >> >> when it
>> > >> >> >> >> >> does:
>> > >> >> >> >> >> >>>> fd = qemu_open("/dev/vfio/vfio", O_RDWR);
>> > >> >> >> >> >> >>>> if (fd < 0) {
>> > >> >> >> >> >> >>>> error_report("vfio: failed to open
>> /dev/vfio/vfio:
>> > >> >> %m");
>> > >> >> >> >> >> >>>> ret = -errno;
>> > >> >> >> >> >> >>>> ...
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>> The rest are followup errors printed from the other
>> > >> >> functions in
>> > >> >> >> >> the
>> > >> >> >> >> >> >>>> stack due to this error.
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>> Bandan
>> > >> >> >> >> >> >>>>
>> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935091Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed to
>> > >> >> >> >> >> >>>>> setup container for group 24
>> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935107Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed to
>> > >> >> >> >> >> >>>>> get group 24
>> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935135Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> Device
>> > >> >> >> >> >> >>>>> initialization failed.
>> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935157Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> Device
>> > >> >> >> >> >> >>>>> 'vfio-pci' could not be initialized
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>> 2015-05-19 21:17 GMT-08:00 Bandan Das <
>> address@hidden>:
>> > >> >> >> >> >> >>>>>
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>>>>> On May 20, 2015, at 12:29 AM, Gabriel Laupre <
>> > >> >> >> address@hidden
>> > >> >> >> >> >
>> > >> >> >> >> >> >>> wrote:
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>> Thank Bandan,
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>> Assuming you are on an intel box, have you
>> booted your
>> > >> >> >> kernel
>> > >> >> >> >> with
>> > >> >> >> >> >> >>>>>> intel_iommu=on ?
>> > >> >> >> >> >> >>>>>>> Yes, I have booted my kernel with the
>> intel_iommu=on.
>> > >> (I
>> > >> >> >> don't
>> > >> >> >> >> >> >>> remember
>> > >> >> >> >> >> >>>>>> how to check that now though ^^)
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>> Please paste the output of dmesg | grep -e DMAR
>> -e
>> > >> IOMMU
>> > >> >> ?
>> > >> >> >> >> >> >>>>>>> address@hidden ~]# dmesg | grep -e DMAR -e IOMMU
>> > >> >> >> >> >> >>>>>>> [ 0.000000] ACPI: DMAR 00000000bf79e0c0 00118
>> (v01
>> > >> >> AMI
>> > >> >> >> >> >> OEMDMAR
>> > >> >> >> >> >> >>>>>> 00000001 MSFT 00000097)
>> > >> >> >> >> >> >>>>>>> [ 0.000000] Intel-IOMMU: enabled
>> > >> >> >> >> >> >>>>>>> [ 0.039149] dmar: IOMMU 0: reg_base_addr
>> fbffe000
>> > >> ver
>> > >> >> 1:0
>> > >> >> >> cap
>> > >> >> >> >> >> >>>>>> c90780106f0462 ecap f020f6
>> > >> >> >> >> >> >>>>>>> [ 0.550126] IOMMU 0 0xfbffe000: using Queued
>> > >> >> invalidation
>> > >> >> >> >> >> >>>>>>> [ 0.550131] IOMMU: Setting RMRR:
>> > >> >> >> >> >> >>>>>>> [ 0.550149] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.0
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550184] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.1
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550211] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.2
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550241] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.7
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550272] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.0
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550302] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.1
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550329] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.2
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550358] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.7
>> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550375] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.0
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550387] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.1
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550399] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.2
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550410] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1a.7
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550421] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.0
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550433] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.1
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550444] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.2
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550458] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1d.7
>> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
>> > >> >> >> >> >> >>>>>>> [ 0.550471] IOMMU: Prepare 0-16MiB unity
>> mapping for
>> > >> >> LPC
>> > >> >> >> >> >> >>>>>>> [ 0.550483] IOMMU: Setting identity map for
>> device
>> > >> >> >> >> 0000:00:1f.0
>> > >> >> >> >> >> >>> [0x0
>> > >> >> >> >> >> >>>>>> - 0xffffff]
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>>>> Yeah, this looks ok. Actually, taking a second
>> look, I
>> > >> >> can’t
>> > >> >> >> >> think
>> > >> >> >> >> >> of
>> > >> >> >> >> >> >>>>>> anyway how this could be related to file
>> permissions on
>> > >> >> >> >> >> /dev/vfio/vfio.
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>>>>>> Why does opening /dev/vfio/vfio fail ? Can you
>> please
>> > >> >> >> confirm
>> > >> >> >> >> that
>> > >> >> >> >> >> >>> you
>> > >> >> >> >> >> >>>>>> have read/write permissions as the user you are
>> trying
>> > >> to
>> > >> >> run
>> > >> >> >> ?
>> > >> >> >> >> >> >>>>>>> address@hidden ~]# cd /dev/vfio/
>> > >> >> >> >> >> >>>>>>> address@hidden vfio]# ls -la | grep vfio
>> > >> >> >> >> >> >>>>>>> crw-rw-rw- 1 root root 10, 196 May 18 11:54 vfio
>> > >> >> >> >> >> >>>>>>> The right should be okay I guess.
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>> Yes, indeed it is. What distro is this ? Do you
>> have
>> > >> >> SELinux
>> > >> >> >> or
>> > >> >> >> >> any
>> > >> >> >> >> >> >>> other
>> > >> >> >> >> >> >>>>>> security feature enabled ? Can you please verify
>> that
>> > >> the
>> > >> >> file
>> > >> >> >> >> has a
>> > >> >> >> >> >> >>>>>> appropriate label if SELinux is enabled ? (ls -lZ
>> > >> >> >> /dev/vfio/vfio)
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>>>> Bandan
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>> 2015-05-19 18:54 GMT-08:00 Bandan Das <
>> address@hidden
>> > >> >:
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>> Hello Gabriel,
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>> On May 19, 2015, at 8:03 PM, Gabriel Laupre <
>> > >> >> >> address@hidden
>> > >> >> >> >> >
>> > >> >> >> >> >> >>> wrote:
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> Hello everyone,
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> I am using a Centos 7.1 machine with the kernel
>> > >> >> 3.10.229. I
>> > >> >> >> >> want
>> > >> >> >> >> >> to
>> > >> >> >> >> >> >>>>>> use my host with SR-IOV to use a virtual function
>> on my
>> > >> >> NIC as
>> > >> >> >> >> the
>> > >> >> >> >> >> >>> vNIC in
>> > >> >> >> >> >> >>>>>> my new VM.
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> I have an instance started with a old NIC using
>> > >> macvtap
>> > >> >> >> that I
>> > >> >> >> >> >> >>> want to
>> > >> >> >> >> >> >>>>>> change. I am using the
>> > >> >> >> >> >> >>>>>>>> virsh edit instance-00000034
>> > >> >> >> >> >> >>>>>>>> command to edit the XML configuration to add the
>> new
>> > >> >> device
>> > >> >> >> I
>> > >> >> >> >> want
>> > >> >> >> >> >> >>> to
>> > >> >> >> >> >> >>>>>> attach.
>> > >> >> >> >> >> >>>>>>> …
>> > >> >> >> >> >> >>>>>>> Assuming you are on an intel box, have you booted
>> your
>> > >> >> kernel
>> > >> >> >> >> with
>> > >> >> >> >> >> >>>>>> intel_iommu=on ?
>> > >> >> >> >> >> >>>>>>> Please paste the output of dmesg | grep -e DMAR -e
>> > >> IOMMU ?
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>> When I try to reboot the VM I get this error:
>> > >> >> >> >> >> >>>>>>>> Error starting domain: internal error: process
>> exited
>> > >> >> while
>> > >> >> >> >> >> >>> connecting
>> > >> >> >> >> >> >>>>>> to monitor: 2015-05-19T21:46:21.935043Z qemu-kvm:
>> > >> -device
>> > >> >> >> >> >> >>>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed
>> > >> >> >> >> >> to
>> > >> >> >> >> >> >>> open
>> > >> >> >> >> >> >>>>>> /dev/vfio/vfio: Operation not permitted
>> > >> >> >> >> >> >>>>>>> Why does opening /dev/vfio/vfio fail ? Can you
>> please
>> > >> >> confirm
>> > >> >> >> >> that
>> > >> >> >> >> >> >>> you
>> > >> >> >> >> >> >>>>>> have read/write permissions as the user you are
>> trying
>> > >> to
>> > >> >> run
>> > >> >> >> ?
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935091Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed
>> > >> >> >> >> >> to
>> > >> >> >> >> >> >>> setup
>> > >> >> >> >> >> >>>>>> container for group 24
>> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935107Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> vfio:
>> > >> >> >> >> failed
>> > >> >> >> >> >> to
>> > >> >> >> >> >> >>> get
>> > >> >> >> >> >> >>>>>> group 24
>> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935135Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> >> Device
>> > >> >> >> >> >> >>> initialization
>> > >> >> >> >> >> >>>>>> failed.
>> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935157Z qemu-kvm: -device
>> > >> >> >> >> >> >>>>>>
>> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
>> > >> >> Device
>> > >> >> >> >> >> 'vfio-pci'
>> > >> >> >> >> >> >>>>>> could not be initialized
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> total Trace here: http://sprunge.us/XZFB
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> Any idea how to fix that?
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> Thank you very much :)
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>> Gabriel
>> > >> >> >> >> >> >>>>>>>>
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>>
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>>>>
>> > >> >> >> >> >> >>>
>> > >> >> >> >> >> >
>> > >> >> >> >> >>
>> > >> >> >> >>
>> > >> >> >>
>> > >> >>
>> > >>
>> >
>> >
>>
>>
| [Prev in Thread] | Current Thread | [Next in Thread] |