Re: [EXT] Re: [PATCH v9 1/9] hw/vfio/common: Remove error print on mmio region translation by viommu

[Auger Eric]

Hi Bharat,

On 5/5/20 11:25 AM, Bharat Bhushan wrote:
> Hi Eric,
> 
> On Fri, Apr 24, 2020 at 7:47 PM Auger Eric <address@hidden> wrote:
>>
>> Hi Bharat,
>>
>> On 4/2/20 11:01 AM, Bharat Bhushan wrote:
>>> Hi Eric/Alex,
>>>
>>>> -----Original Message-----
>>>> From: Alex Williamson <address@hidden>
>>>> Sent: Thursday, March 26, 2020 11:23 PM
>>>> To: Auger Eric <address@hidden>
>>>> Cc: Bharat Bhushan <address@hidden>; address@hidden;
>>>> address@hidden; address@hidden; address@hidden;
>>>> address@hidden; Tomasz Nowicki [C] <address@hidden>;
>>>> address@hidden; address@hidden; address@hidden; qemu-
>>>> address@hidden; address@hidden; address@hidden;
>>>> address@hidden; David Gibson <address@hidden>
>>>> Subject: [EXT] Re: [PATCH v9 1/9] hw/vfio/common: Remove error print on 
>>>> mmio
>>>> region translation by viommu
>>>>
>>>> External Email
>>>>
>>>> ----------------------------------------------------------------------
>>>> On Thu, 26 Mar 2020 18:35:48 +0100
>>>> Auger Eric <address@hidden> wrote:
>>>>
>>>>> Hi Alex,
>>>>>
>>>>> On 3/24/20 12:08 AM, Alex Williamson wrote:
>>>>>> [Cc +dwg who originated this warning]
>>>>>>
>>>>>> On Mon, 23 Mar 2020 14:16:09 +0530
>>>>>> Bharat Bhushan <address@hidden> wrote:
>>>>>>
>>>>>>> On ARM, the MSI doorbell is translated by the virtual IOMMU.
>>>>>>> As such address_space_translate() returns the MSI controller MMIO
>>>>>>> region and we get an "iommu map to non memory area"
>>>>>>> message. Let's remove this latter.
>>>>>>>
>>>>>>> Signed-off-by: Eric Auger <address@hidden>
>>>>>>> Signed-off-by: Bharat Bhushan <address@hidden>
>>>>>>> ---
>>>>>>>  hw/vfio/common.c | 2 --
>>>>>>>  1 file changed, 2 deletions(-)
>>>>>>>
>>>>>>> diff --git a/hw/vfio/common.c b/hw/vfio/common.c index
>>>>>>> 5ca11488d6..c586edf47a 100644
>>>>>>> --- a/hw/vfio/common.c
>>>>>>> +++ b/hw/vfio/common.c
>>>>>>> @@ -426,8 +426,6 @@ static bool vfio_get_vaddr(IOMMUTLBEntry *iotlb,
>>>> void **vaddr,
>>>>>>>                                   &xlat, &len, writable,
>>>>>>>                                   MEMTXATTRS_UNSPECIFIED);
>>>>>>>      if (!memory_region_is_ram(mr)) {
>>>>>>> -        error_report("iommu map to non memory area %"HWADDR_PRIx"",
>>>>>>> -                     xlat);
>>>>>>>          return false;
>>>>>>>      }
>>>>>>>
>>>>>>
>>>>>> I'm a bit confused here, I think we need more justification beyond
>>>>>> "we hit this warning and we don't want to because it's ok in this
>>>>>> one special case, therefore remove it".  I assume the special case
>>>>>> is that the device MSI address is managed via the SET_IRQS ioctl and
>>>>>> therefore we won't actually get DMAs to this range.
>>>>> Yes exactly. The guest creates a mapping between one giova and this
>>>>> gpa (corresponding to the MSI controller doorbell) because MSIs are
>>>>> mapped on ARM. But practically the physical device is programmed with
>>>>> an host chosen iova that maps onto the physical MSI controller's
>>>>> doorbell. so the device never performs DMA accesses to this range.
>>>>>
>>>>>   But I imagine the case that
>>>>>> was in mind when adding this warning was general peer-to-peer
>>>>>> between and assigned and emulated device.
>>>>> yes makes sense.
>>>>>
>>>>>   Maybe there's an argument to be made
>>>>>> that such a p2p mapping might also be used in a non-vIOMMU case.  We
>>>>>> skip creating those mappings and drivers continue to work, maybe
>>>>>> because nobody attempts to do p2p DMA with the types of devices we
>>>>>> emulate, maybe because p2p DMA is not absolutely reliable on bare
>>>>>> metal and drivers test it before using it.
>>>>> MSI doorbells are mapped using the IOMMU_MMIO flag (dma-iommu.c
>>>>> iommu_dma_get_msi_page).
>>>>> One idea could be to pass that flag through the IOMMU Notifier
>>>>> mechanism into the iotlb->perm. Eventually when we get this in
>>>>> vfio_get_vaddr() we would not print the warning. Could that make sense?
>>>>
>>>> Yeah, if we can identify a valid case that doesn't need a warning, that's 
>>>> fine by me.
>>>> Thanks,
>>>
>>> Let me know if I understood the proposal correctly:
>>>
>>> virtio-iommu driver in guest will make map (VIRTIO_IOMMU_T_MAP) with 
>>> VIRTIO_IOMMU_MAP_F_MMIO flag for MSI mapping.
>>> In qemu, virtio-iommu device will set a new defined flag (say IOMMU_MMIO) 
>>> in iotlb->perm in memory_region_notify_iommu(). vfio_get_vaddr() will check 
>>> same flag and will not print the warning.>
>>> Is above correct?
>> Yes that's what I had in mind.
> 
> In that case virtio-iommu driver in guest should not make map
> (VIRTIO_IOMMU_T_MAP) call as it known nothing to be mapped.
sorry I don't catch what you meant. Please can you elaborate?

Thanks

Eric
> 
> Stay Safe
> 
> Thanks
> -Bharat
> 
>>
>> Thanks
>>
>> Eric
>>>
>>> Thanks
>>> -Bharat
>>>
>>>>
>>>> Alex
>>>
>>>
>>
>