Re: [Qemu-devel] [PATCH 09/13] iommu: Add facility to cancel in-use dma memory maps

[Anthony Liguori]

On 06/19/2012 01:39 AM, Benjamin Herrenschmidt wrote:
> From: David Gibson<address@hidden>
>
> One new complication raised by IOMMU support over only handling DMA
> directly to physical addresses is handling dma_memory_map() case
> (replacing cpu_physical_memory_map()) when the IOMMU translation the
> IOVAs covered by such a map are invalidated or changed while the map
> is active.  This should never happen with correct guest software, but
> we do need to handle buggy guests.  This case might also occur during
> handovers between different guest software stages if the handover
> protocols aren't fully seamless.
>
> The iommu implementation will have to wait for maps to be removed
> before it can "complete" an invalidation of a translation, which
> can take a long time. In order to make it possible to speed that
> process up, we add a "Cancel" callback to the map function which
> the clients can optionally provide.
>
> The core makes no use of that, but the iommu backend implementation
> may choose to keep track of maps and call the respective cancel
> callback whenever a translation within a map is removed, allowing
> the driver to do things like cancel async IOs etc.
>
> Signed-off-by: David Gibson<address@hidden>
> Signed-off-by: Benjamin Herrenschmidt<address@hidden>
> ---
>   dma-helpers.c |   49 ++++++++++++++++++++++++++++---------------------
>   dma.h         |   23 +++++++++++++++++++----
>   2 files changed, 47 insertions(+), 25 deletions(-)
>
> diff --git a/dma-helpers.c b/dma-helpers.c
> index b4ee827..6e6c7b3 100644
> --- a/dma-helpers.c
> +++ b/dma-helpers.c
> @@ -107,6 +107,28 @@ static void dma_complete(DMAAIOCB *dbs, int ret)
>       }
>   }
>
> +static void dma_aio_cancel(BlockDriverAIOCB *acb)
> +{
> +    DMAAIOCB *dbs = container_of(acb, DMAAIOCB, common);
> +
> +    trace_dma_aio_cancel(dbs);
> +
> +    if (dbs->acb) {
> +        BlockDriverAIOCB *acb = dbs->acb;
> +        dbs->acb = NULL;
> +        dbs->in_cancel = true;
> +        bdrv_aio_cancel(acb);
> +        dbs->in_cancel = false;
> +    }
> +    dbs->common.cb = NULL;
> +    dma_complete(dbs, 0);

So this cancellation stuff is hopelessly broken

It's simply not possible to fully cancel pending DMA in a synchronous callback.

Indeed, bdrv_aio_cancel ends up having a nasty little loop in it:

    if (active) {
        /* fail safe: if the aio could not be canceled, we wait for
           it */
        while (qemu_paio_error(acb) == EINPROGRESS)
            ;
    }

That spins w/100% CPU.

Can you explain when DMA cancellation really happens and what the effect would 
be if we simply ignored it?

Can we do something more clever like use an asynchronous callback to handle 
flushing active DMA mappings?

There's just no way a callback like this is going to work.

Regards,

Anthony Liguori