Re: [Qemu-ppc] KVM and variable-endianness guest CPUs

[Victor Kamensky]

On 22 January 2014 12:02, Peter Maydell <address@hidden> wrote:
> On 22 January 2014 19:29, Victor Kamensky <address@hidden> wrote:
>> On 22 January 2014 09:29, Peter Maydell <address@hidden> wrote:
>>> This just isn't how real buses work. There is no
>>> "address + 1, address + 2". There is a single address
>>> for the memory transaction and a set of data on
>>> data lines and some separate size information.
>>
>> Yes, and those data lines are just binary signal lines
>> not numbers. If one would want to describe information
>> on data lines as number he/she needs to assign integer
>> bits numbers to lines, and that is absolutely arbitrary
>> process
>
> It is part of the definition of the bus which signal pin is
> D0 and which is D31...
>
>>  In one choose one way to assigned those
>> bits to lines and another choose reverse way, they will
>> talk about completely different numbers for the same
>> signals on the bus. Such data lines enumeration has
>> no reflection on how bus actually works. And I don't
>> even see why it should be described just as single
>> integer, for example one can describe information on
>> data lines as set of 4 byte value, nothing wrong with
>> such description.
>
> It is not how the hardware works. If you describe it as
> a set of 4 bytes, then you need to also say how you are
> mapping from those 4 bytes to the actual 32 bit data
> transaction the hardware is doing. Which is the question
> we're trying to answer in this thread.
>
> I've snipped a huge chunk of my initial reply to this email,
> because it all boiled down to "sorry, you're just not correct
> about how the hardware works" and it doesn't seem
> necessary to repeat it three times. Devices really do see
> "this is a transaction with this value and this size". They
> do not in any way see a 32 bit word write as "this is a collection
> of byte writes". Therefore:
>
>  1) thinking about a 32 bit word write in terms of a byte array
>     is confusing
>  2) since the KVM API is unfortunately stuck with this byte
>    array, we must define the semantics of what it actually
>    contains, so that the kernel and QEMU can go between
>   "the value being read/written in the transaction" and
>   "the contents of the byte array
>
>>> As soon as you try to think of the mmio.data as a set
>>> of bytes then you have to specify some endianness to
>>> the data, so that both sides (kernel and userspace)
>>> know how to reconstruct the actual data value from the
>>> array of bytes.
>>
>> What actual value? In what sense? You need to bring
>> into discussion semantic of this h/w address to really tell
>> that.
>
> I've just spent the last two emails doing exactly that.
> The actual value, as in "this CPU just did a memory
> transaction of a 32 bit data value".

You deleted my example, but I need it again:
Consider the following ARM code snippets:

setend le
mov r1, #0x04030201
str r1, [r0]

and

setend be
mov r1, #0x01020304
str r1, [r0]

Just for LE host case basically you are saying that if guest
issues 4 bytes store
instruction for CPU core register and CPSR E bit is off,
mmio.data[0] would contain LSB of integer from this CPU core
register. I don't understand your bus endianity thing, but I do
understand LSB of integer in core CPU register. Do we agree
that in above example in second case when BE access is
on (E bit is on), it is the exactly the same memory transaction
but it data[0] = 0x1 is MSB of integer in CPU core register (still the
same LE host case)?

>> BTW could you please propose how will you see such
>> "32 bit transaction, value 0x04030201, address $whatever".
>> on ARM LE CPU in mmio.data?
>
> That is exactly the problem we're discussing in this thread.
> Indeed, I proposed an answer to it, which is that the mmio.data
> array should be in host kernel byte order, in which case it
> would be (for an LE host kernel) 0x01 in mmio.data[0] and so
> on up.
>
>> If it would be {0x01, 0x02, 0x03, 0x4} it is fine with
>> me. That is current case ARM LE case when above
>> snippets would be executed by guest.
>>
>> Would we  agree that the same arrangement would be
>> true for all other cases on ARM regardless of all other
>> endianities of qemu, KVM host, guest, hypervisor, etc?
>
> No; under my proposal, for a big-endian host kernel (and
> thus big-endian QEMU) the order would be
> mmio.data[0] = 0x04, etc. (This wouldn't change based
> on the guest kernel endianness or whether it happened
> to have set the E bit temporarily.)

Consider above big endian case (setend be) example,
but now running in BE KVM host. 0x4 is LSB of CPU
core register in this case.

> Defining that mmio.data[] is always little-endian would
> be a valid definition of an API if we were doing it from
> scratch. It has the unfortunate property that it would
> completely break the existing PPC BE setups, which
> don't define it that way, so it is a non-starter.

I believe, but I need to check, that PPC BE setup actually
acts as the second case in above example  If we have PPC
BE guest executing the following instructions:

lis     r1,0x102
ori     r1,r1,0x304
stw    r1,0(r0)

after first two instructions r1 would contain 0x01020304.
IMHO It exactly corresponds to above my ARM second case -
BE guest when it runs under ARM BE KVM host. I believe
that mmio.data[] in PPC BE case would be {0x1, 0x2, 0x3, 0x4}.
That would be the same as I argue that should be the case for
both ARM and PPC for any choice of endianities of KVM host,
and emulator.

But according to you data[0] must be 0x4 in BE host case and
If you are right and in PPC BE case for above ppc instructions
sequence executed in BE mode, running on BE host
mmio.data[] = {0x04, 0x03, 0x02, 0x01}, i.e integer at &mmio.data[0]
address has LE format from from original CPU core register integer
point of view, my bytes copy interpretation of mmio.data[]
indeed falls to pieces. Such PPC case breaks it.

Could someone on these mailing lists quickly check PPC case
with  similar as above guest instructions sequence. I could try to
do it myself, but I think it may take me couple days to set this
up .. need to dust off my p5020 board :) ...

Thanks,
Victor

> Defining it as being always guest-order would mean that
> userspace had to continually look at the guest CPU
> endianness bit, which is annoying and awkward.
>
> Defining it as always host-endian order is the most
> reasonable option available. It also happens to work
> for the current QEMU code, which is nice.
>
> thanks
> -- PMM