Re: [Qemu-devel] [PATCH v4 0/9] enable numa configuration before machine_init() from QMP

[Markus Armbruster]

Eduardo Habkost <address@hidden> writes:

> On Tue, Apr 17, 2018 at 05:41:10PM +0200, Igor Mammedov wrote:
>> On Tue, 17 Apr 2018 11:27:39 -0300
>> Eduardo Habkost <address@hidden> wrote:
>> 
>> > On Tue, Apr 17, 2018 at 04:13:34PM +0200, Markus Armbruster wrote:
>> > > Igor Mammedov <address@hidden> writes:
>> > > 
>> > > [...]  
>> > > > Series allows to configure NUMA mapping at runtime using QMP
>> > > > interface. For that to happen it introduces a new '-preconfig' CLI 
>> > > > option
>> > > > which allows to pause QEMU before machine_init() is run and
>> > > > adds new set-numa-node QMP command which in conjunction with
>> > > > query-hotpluggable-cpus allows to configure NUMA mapping for cpus.
>> > > >
>> > > > Later we can modify other commands to run early, for example 
>> > > > device_add.
>> > > > I recall SPAPR had problem when libvirt started QEMU with -S and, 
>> > > > while it's
>> > > > paused, added CPUs with device_add. Intent was to coldplug CPUs (but 
>> > > > at that
>> > > > stage it's considered hotplug already), so SPAPR had to work around 
>> > > > the issue.  
>> > > 
>> > > That instance is just stupidity / laziness, I think: we consider any
>> > > plug after machine creation a hot plug.  Real machines remain cold until
>> > > you press the power button.  Our virtual machines should remain cold
>> > > until they start running, i.e. with -S until the first "cont".
>> It probably would be too risky to change semantics of -S from hotplug to 
>> coldplug.
>> But even if we were easy it won't matter in case if dynamic configuration
>> done properly. More on it below.
>> 
>> > > I vaguely remember me asking this before, but your answer didn't make it
>> > > into this cover letter, which gives me a pretext to ask again instead of
>> > > looking it up in the archives: what exactly prevents us from keeping the
>> > > machine cold enough for numa configuration until the first "cont"?  
>> > 
>> > I also think this would be better, but it seems to be difficult
>> > in practice, see:
>> > http://mid.mail-archive.com/address@hidden
>> 
>> In addition to Eduardo's reply, here is what I've answered back
>> when you've asked question the 1st time (v2 late at -S pause point reconfig):
>> https://www.mail-archive.com/address@hidden/msg504140.html
>> 
>> In short:
>> I think it's wrong in general doing fixups after machine is build
>> instead of getting correct configuration before building machine.
>> That's going to be complex and fragile and might be hard to do at
>> all depending on what we are fixing up.
>
> What "building the machine" should mean, exactly, for external
> users?
>
> The main question I'd like to see answered is: why exactly we
> must "build" the machine before the first "cont" is issued when
> using -S?  Why can't we delay everything to "cont" when using -S?

Exactly.

> Is it just because it's a long and complex task?  Does that mean
> we might still do that eventually, and eliminate the
> prelaunch/preconfig distinction in the distant future?

Why would anyone want to use -S going forward?  For reasons other "we've
always used -S, and can't be bothered to change".

> Even if we follow your approach, we need to answer these
> questions.  I'm sure we will try to reorder initialization steps
> between the preconfig/prelaunch states in the future, and we
> shouldn't break any expectations from external users when doing
> that.

Moreover, the questions need to be answered in Git.  Commit message,
comments, docs/, use your judgement.

>> BTW this is an outdated version of series and there is a newer one v5
>> https://patchwork.ozlabs.org/cover/895315/

Sorry about that.  I'm drowning in a sea of two months worth of patches.

>> so pleases review it.
>> 
>> Short diff vs 1:
>>  - only limited(minimum) set of commands is available at preconfig stage for 
>> now
>>  - use QAPI schema to mark commands as preconfig enabled,
>>    so mgmt could see when it can use commands.
>>  - added preconfig runstate state-machine instead of adding more global 
>> variables
>>    to cleanly keep track of where QEMU is paused and what it's allowed to do