Re: [PATCH v3 00/13] vfio/migration: Device dirty page tracking

[Joao Martins]

On 06/03/2023 02:19, Alex Williamson wrote:
> On Sun, 5 Mar 2023 23:33:35 +0000
> Joao Martins <joao.m.martins@oracle.com> wrote:
> 
>> On 05/03/2023 20:57, Alex Williamson wrote:
>>> On Sat,  4 Mar 2023 01:43:30 +0000
>>> Joao Martins <joao.m.martins@oracle.com> wrote:
>>>   
>>>> Hey,
>>>>
>>>> Presented herewith a series based on the basic VFIO migration protocol v2
>>>> implementation [1].
>>>>
>>>> It is split from its parent series[5] to solely focus on device dirty
>>>> page tracking. Device dirty page tracking allows the VFIO device to
>>>> record its DMAs and report them back when needed. This is part of VFIO
>>>> migration and is used during pre-copy phase of migration to track the
>>>> RAM pages that the device has written to and mark those pages dirty, so
>>>> they can later be re-sent to target.
>>>>
>>>> Device dirty page tracking uses the DMA logging uAPI to discover device
>>>> capabilities, to start and stop tracking, and to get dirty page bitmap
>>>> report. Extra details and uAPI definition can be found here [3].
>>>>
>>>> Device dirty page tracking operates in VFIOContainer scope. I.e., When
>>>> dirty tracking is started, stopped or dirty page report is queried, all
>>>> devices within a VFIOContainer are iterated and for each of them device
>>>> dirty page tracking is started, stopped or dirty page report is queried,
>>>> respectively.
>>>>
>>>> Device dirty page tracking is used only if all devices within a
>>>> VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is
>>>> used, and if that is not supported as well, memory is perpetually marked
>>>> dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW
>>>> support, the last two usually have the same effect of perpetually
>>>> marking all pages dirty.
>>>>
>>>> Normally, when asked to start dirty tracking, all the currently DMA
>>>> mapped ranges are tracked by device dirty page tracking. If using a
>>>> vIOMMU we block live migration. It's temporary and a separate series is
>>>> going to add support for it. Thus this series focus on getting the
>>>> ground work first.
>>>>
>>>> The series is organized as follows:
>>>>
>>>> - Patches 1-7: Fix bugs and do some preparatory work required prior to
>>>>   adding device dirty page tracking.
>>>> - Patches 8-10: Implement device dirty page tracking.
>>>> - Patch 11: Blocks live migration with vIOMMU.
>>>> - Patches 12-13 enable device dirty page tracking and document it.
>>>>
>>>> Comments, improvements as usual appreciated.  
>>>
>>> Still some CI failures:
>>>
>>> https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474
>>>
>>> The docker failures are normal, afaict the rest are not.  Thanks,
>>>   
>>
>> Ugh, sorry
>>
>> The patch below scissors mark (and also attached as a file) fixes those build
>> issues. I managed to reproduce on i386 target builds, and these changes fix 
>> my
>> 32-bit build.
>>
>> I don't have a working Gitlab setup[*] though to trigger the CI to enable to
>> wealth of targets it build-tests. If you could kindly test the patch 
>> attached in
>> a new pipeline (applied on top of the branch you just build) below to 
>> understand
>> if the CI gets happy. I will include these changes in the right patches 
>> (patch 8
>> and 10) for the v4 spin.
> 
> Looks like this passes:
> 
> https://gitlab.com/alex.williamson/qemu/-/pipelines/796750136
> 
Great, I've staged this fixes in patches 8&10!

I have a sliver of hope that we might still make it by soft freeze (tomorrow?).
If you think it can still make it, should the rest of the series is good, then I
can follow up v4 today/tomorrow. Thoughts?

        Joao