Re: [PATCH 3/3] pci: ROM preallocation for incoming migration

[Vladimir Sementsov-Ogievskiy]

On 25.04.23 17:55, Igor Mammedov wrote:
> On Tue, 25 Apr 2023 09:32:54 -0400
> "Michael S. Tsirkin"<mst@redhat.com>  wrote:
>
> > On Tue, Apr 25, 2023 at 04:19:12PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > > On 25.04.23 16:07, Vladimir Sementsov-Ogievskiy wrote:
> > > > On 25.04.23 15:43, Michael S. Tsirkin wrote:
> > > > > On Tue, Apr 25, 2023 at 01:56:03PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > > > > > On incoming migration we have the following sequence to load option
> > > > > > ROM:
> > > > > >
> > > > > > 1. On device realize we do normal load ROM from the file
> > > > > >
> > > > > > 2. Than, on incoming migration we rewrite ROM from the incoming RAM
> > > > > >      block. If sizes mismatch we fail.
> > > > > >
> > > > > > This is not ideal when we migrate to updated distribution: we have to
> > > > > > keep old ROM files in new distribution and be careful around romfile
> > > > > > property to load correct ROM file. Which is loaded actually just to
> > > > > > allocate the ROM with correct length.
> > > > > >
> > > > > > Note, that romsize property doesn't really help: if we try to specify
> > > > > > it when default romfile is larger, it fails with something like:
> > > > > >
> > > > > > romfile "efi-virtio.rom" (160768 bytes) is too large for ROM size 65536
> > > > > >
> > > > > > This commit brings new behavior for romfile="",romsize=SIZE combination
> > > > > > of options. Current behavior is just ignore romsize and not load or
> > > > > > create any ROM.
> > > > > >
> > > > > > Let's instead preallocate ROM, not loading any file. This way we can
> > > > > > migrate old vm to new environment not thinking about ROM files on
> > > > > > destination host:
> > > > > >
> > > > > > 1. specify romfile="",romsize=SIZE on target, with correct SIZE
> > > > > >      (actually, size of romfile on source aligned up to power of two, or
> > > > > >       just original romsize option on source)
> > > > > >
> > > > > > 2. On device realize we just preallocate ROM, and not load any file
> > > > > >
> > > > > > 3. On incoming migration ROM is filled from the migration stream
> > > > > >
> > > > > > As a bonus we avoid extra reading from ROM file on target.
> > > > > >
> > > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy<vsementsov@yandex-team.ru>   
> > > > > why is this a bad idea:
> > > > > - on source presumably user overrides romfile
> > > > > - we have a general rule that source and destination flags must match
> > > > >
> > > > > I propose instead to ignore romfile if qemu is incoming migration
> > > > > and romsize has been specified.
> > > > >    
> > > > Hmm, that would work even better, as no additional options needed, thanks. I'll 
> > > > resend
> > > >    
> > > romsize needed anyway, of course.
> > yes but it can match on source and dest.
> Aren't we pushin issue from QEMU(/how distro packages firmware/)
> to mgmt layer(s) going this way?

I'm afraid, we can't simply solve the issue on QEMU part, as we need this 
romsize at realize time. If we want to change the size on load of incoming 
migration, we'd have to reinitialize the device and memory layout.. This seems 
too difficult.

On the other hand, it seems correct to force management to specify ROM size to 
create. Same as RAM size.

Imagine, we always had the following behavior:

 - romsize is necessary parameter when we have rom (absence means no rom, same as 
romfile="")
 - romfile is ignored on incoming migration

this way, we would never have any migration problems.

So, the proposal is to at least support such behavior:

 - ignore romfile on incoming migration when romsize is specified.

Additionally we can deprecate missing romsize, when we have rom.


--
Best regards,
Vladimir