[Qemu-devel] Re: [PATCHv4 15/15] Pass boot device list to firmware.

[Gleb Natapov]

On Thu, Nov 18, 2010 at 02:23:20PM +0200, Michael S. Tsirkin wrote:
> On Thu, Nov 18, 2010 at 02:16:05PM +0200, Gleb Natapov wrote:
> > On Thu, Nov 18, 2010 at 01:52:30PM +0200, Michael S. Tsirkin wrote:
> > > On Thu, Nov 18, 2010 at 01:45:04PM +0200, Gleb Natapov wrote:
> > > > On Thu, Nov 18, 2010 at 01:38:31PM +0200, Michael S. Tsirkin wrote:
> > > > > On Thu, Nov 18, 2010 at 12:18:27PM +0200, Gleb Natapov wrote:
> > > > > > On Wed, Nov 17, 2010 at 09:54:27PM +0000, Blue Swirl wrote:
> > > > > > > 2010/11/16 Gleb Natapov <address@hidden>:
> > > > > > > > On Tue, Nov 16, 2010 at 06:30:19PM +0000, Blue Swirl wrote:
> > > > > > > >> >> Perhaps the FW path should use device class names if no 
> > > > > > > >> >> name is specified.
> > > > > > > >> > What do you mean by "device class name". We can do something 
> > > > > > > >> > like this:
> > > > > > > >> > if (dev->child_bus.lh_first)
> > > > > > > >> >        return dev->child_bus.lh_first->info->name;
> > > > > > > >> >
> > > > > > > >> > i.e if there is child bus use its bus name as fw name. This 
> > > > > > > >> > will make
> > > > > > > >> > all pci devices to have "pci" as fw name automatically. The 
> > > > > > > >> > problem is
> > > > > > > >> > that theoretically same device can provide different buses.
> > > > > > > >>
> > > > > > > >> I meant PCI class name, like "display" for display controllers,
> > > > > > > >> "network" for NICs etc.
> > > > > > > >>
> > > > > > > > That is what my pci bus related patch is doing already.
> > > > > > > >
> > > > > > > >> >> I'll try Sparc32 to see how this fits there.
> > > > > > > >>
> > > > > > > >> Except bootindex is not implemented for SCSI.
> > > > > > > > Will look into adding it.
> > > > > > > 
> > > > > > > Thanks. The bootindex on Sparc32 looks like this:
> > > > > > > bootindex /address@hidden/address@hidden,0
> > > > > > > /address@hidden/address@hidden
> > > > > > > 
> > > > > > For arches other then x86 there is a lot of work left to be done :)
> > > > > > For starter exotic sparc buses should get their own 
> > > > > > get_fw_dev_path()
> > > > > > implementation.
> > > > > > 
> > > > > > > I don't think I got Lance setup right.
> > > > > > > 
> > > > > > > OF paths for the devices would be:
> > > > > > > /address@hidden,10000000/address@hidden,10001000/address@hidden,8400000/address@hidden,8800000/address@hidden,0
> > > > > > > /address@hidden,10000000/address@hidden,10001000/address@hidden,8400010/address@hidden,8c00000
> > > > > > If qdev hierarchy does not correspond to real HW there is no much 
> > > > > > we can
> > > > > > do expect for fixing qdev.
> > > > > 
> > > > > That's bad.  This raises a concern: if these paths expose qdev
> > > > > internals, any attempt to fix this will break migration.
> > > > > 
> > > > The path expose internal HW hierarchy. It is designed to do so. Qdev
> > > > designed to do the same: describe HW hierarchy. If qdev fails to do so 
> > > > it
> > > > is broken.
> > > 
> > > Yes. But since you use qdev to build up the path, a broken
> > > qdev will give you a broken path.
> > > 
> > Qdev bug. Fix it like any other bug. The nice is that when you compare
> > device path produced by qdev and real HW you can see when qdev is wrong.
> > 
> > > > I do not see connection to migration at all since the path is
> > > > not used in migration code.
> > > 
> > > The connection is that if we pass the list with path 1 which you define
> > > as broken to BIOS, then migrate to a machine with an updated qemu
> > > which has a correct path, BIOS won't be able to complete the boot.
> > You solve it like you solve all such issue with -M machine type.
> 
> So that's unavoidable if we think paths are correct.
> But if we know they are wrong, we are better off
> correcting them first IMO.
> 
They are correct for x86. My patch set does not even tries to cover all
HW. If sparc want to use them to it better be fixed. Or if there is enough
info in the path to determine device it may choose to use it as is.

> > But the problem exists only if migration happens in a short window
> > between start of the boot process and BIOS reading boot order string.
> > After reboot new qemu should have new BIOS.
> 
> That makes it even more nasty, doesn't it?
No.

> 
> > > Right? Same in reverse direction.
> > Reverse direction is not and never was supported.
> > 
> > > As solution could be a fuzzy matching
> > > of paths that wiull let us recover.
> > > 
> > Firmware can try its best of course, but nothing is guarantied.
> 
> No I mean qemu could do matching fuzzily.
> This way if we get a path from the old BIOS we can
> survive.
Qemu does not take paths from BIOS so I don't know what are you talking
about here.

> 
> > > > > > > 
> > > > > > > The logic for ESP is that ESP (registers at 0x78800000, slot 
> > > > > > > offset
> > > > > > > 0x880000) is handled by the DMA controller (registers at 
> > > > > > > 0x78400000,
> > > > > > > slot offset 0x840000), they are in a SBus slot #5, and SBus 
> > > > > > > (registers
> > > > > > > at 0x10001000) is in turn handled by IOMMU (registers at 
> > > > > > > 0x10000000).
> > > > > > > Lance should be handled the same way.
> > > > > > > 
> > > > > > > This hierarchy is partly known by QEMU because DMA accesses use 
> > > > > > > this
> > > > > > > flow, but not otherwise. There is no concept of SBus slots, DMA 
> > > > > > > talks
> > > > > > > to IOMMU directly. Though in this case both ESP, Lance and their 
> > > > > > > DMA
> > > > > > > controllers are on board devices in a MACIO chip. It may be 
> > > > > > > possible
> > > > > > > to add the hierarchy information at each stage.
> > > > > > > 
> > > > > > > It should also be possible for BIOS to determine the device just 
> > > > > > > from
> > > > > > > the physical address if we ignored OF compatibility.
> > > > > > It would be nice to be OF compatible at least at some level. Of 
> > > > > > course OF
> > > > > > spec is not strict enough to have two different implementations 
> > > > > > produce
> > > > > > exactly same device path that can be compared by strcpy.  Can we 
> > > > > > apply
> > > > > > the series now? At least for x86 it provides useful paths and work 
> > > > > > can
> > > > > > be continue for other arches by interested parties.
> > > > > > 
> > > > > > --
> > > > > >                     Gleb.
> > > > > 
> > > > > Something I only now realized is that we commit
> > > > > to never changing the paths for any architecture
> > > > > that supports migration.
> > > > > 
> > > > No connection to migration whatsoever.
> > > 
> > > It just seems silly to use different paths for the same thing.
> > > 
> > > Besides the connection above, I was hoping to use these paths
> > > for section names in migration. If we can't guarantee they are
> > > stable, we'll have to roll our own, and if we do this,
> > > with stability guarantees required for migration format,
> > > maybe use it for other things like BIOS as well?
> > > 
> > It doesn't matter what do you use for migration purposes as long as it
> > depend on qdev hierarchy it will have problem when qdev hierarchy
> > changes and if it doesn't you can't produce unique names reliably.
> > 
> > --
> >                     Gleb.
> 
> We can, it's not like OF is the only way to enumerate.
> We could have driver-specific paths for example, exactly like we
> currently have.  I.e. paths don't have to be globally unique because
> each driver has it's own domain.
> 
What we have currently is not even close to be correct. It happens to
work since it is implemented only for one bus type and we can have only
one of this bus right now. And of course it is not suitable for passing
boot path to FW since it does not describe path to device.


> It seems cleaner to use an existing spec but we must figure out
> how it will not become a support issue.
> 
If you think you can figure out how to describe device path (or even
give globally unique name to device) and to not depend on internal qdev
implementation go ahead and do that for migration.

--
                        Gleb.