Re: [PATCH v2 2/5] apic: add support for x2APIC mode

[Bui Quang Minh]

On 4/3/23 17:27, David Woodhouse wrote:
> On Wed, 2023-03-29 at 22:30 +0700, Bui Quang Minh wrote:
> > > I do some more testing on my hardware, your point is correct when dest
> > > == 0xffffffff, the interrupt is delivered to all APICs regardless of
> > > their mode.
> >
> > To be precisely, it only broadcasts to CPUs in xAPIC mode if the IPI
> > destination mode is physical. In case the destination mode is logical,
> > flat model/cluster model rule applies to determine if the xAPIC CPUs
> > accept the IPI. Wow, this is so complicated :)
>
> So even if you send to *all* of the first 8 CPUs in a cluster (e.g.
> cluster 0x0001 giving a destination 0x000100FF, a CPU in xAPIC mode
> doesn't see that as a broadcast because it's logical mode?

I mean if the destination is 0xffffffff, the xAPIC CPU will see 
destination as 0xff. 0xff is broadcast in physical destination mode 
only, in logical destination, it may not be a broadcast. It may depend 
on the whether it is flat model or cluster model in logical destination 
mode.

In flat model, 8 bits are used as mask, so in theory, this model can 
only support 8 CPUs, each CPU reserves its own bit by setting the upper 
8 bits of APIC LDR register. In Intel SDM, it is said that 0xff can be 
interpreted as a broadcast, this is true in normal case, but I think if 
the CPU its APIC LDR to 0, the IPI should not deliver to this CPU. This 
also matches with the current flat model of logical destination mode 
implementation of userspace APIC in Qemu before my series. However, I 
see this seems like a corner case, I didn't test it on real hardware.

With cluster model, when writing the above paragraphs, I think that 0xff 
will be delivered to all APICs (mask = 0xf) of cluster 15 (0xf). 
However, reading the SDM more carefully, I see that the there are only 
15 clusters with address from 0 to 14 so there is no cluster with 
address 15. 0xff is interpreted as broadcast to all APICs in all 
clusters too.

In conclusion, IPI with destination 0xffffffff can be a broadcast to all 
xAPIC CPUs too if we just ignore the corner case in flat model of 
logical destination mode (we may need to test more)

> I would have assumed that a CPU in xAPIC mode would have looked at the
> low byte and interpreted it as xAPIC logical mode, with the cluster in
> the high nybble and the 4-bit mask in the low nybble?

Yes, this is the behavior in cluster model of logical destination mode 
(I try to stick with Intel SDM Section 10.6.2.2 Volume 3A when using 
those terminologies)