Re: [Qemu-ppc] KVM and variable-endianness guest CPUs

[Victor Kamensky]

On 22 January 2014 09:29, Peter Maydell <address@hidden> wrote:
> On 22 January 2014 17:19, Victor Kamensky <address@hidden> wrote:
>> On 22 January 2014 02:22, Peter Maydell <address@hidden> wrote:
>>> but the major issue here is that the data being
>>> transferred is not just a bag of bytes. The data[]
>>> array plus the size field are being (mis)used to indicate
>>> that the memory transaction is one of:
>>>  * an 8 bit access
>>>  * a 16 bit access of some uint16_t value
>>>  * a 32 bit access of some uint32_t value
>>>  * a 64 bit access of some uint64_t value
>>>
>>> exactly as a CPU hardware bus would do. It's
>>> because the API is defined in this awkward way with
>>> a uint8_t[] array that we need to specify how both
>>> sides should go from the actual properties of the
>>> memory transaction (value and size) to filling in the
>>> array.
>>
>> While responding to Alex last night, I found, I think,
>> easiest and shortest way to think about mmio.data[]
>>
>> Just for discussion reference here it is again
>>                 struct {
>>                         __u64 phys_addr;
>>                         __u8  data[8];
>>                         __u32 len;
>>                         __u8  is_write;
>>                 } mmio;
>> I believe that in all cases it should be interpreted
>> in the following sense
>>    byte data[0] goes into byte at phys_addr + 0
>>    byte data[1] goes into byte at phys_addr + 1
>>    byte data[2] goes into byte at phys_addr + 2
>>    and so on up to len size
>>
>> Basically if it would be on real bus, get byte value
>> that corresponds to phys_addr + 0 address place
>> it into data[0], get byte value that corresponds to
>> phys_addr + 1 address place it into data[1], etc.
>
> This just isn't how real buses work. There is no
> "address + 1, address + 2". There is a single address
> for the memory transaction and a set of data on
> data lines and some separate size information.

Yes, and those data lines are just binary signal lines
not numbers. If one would want to describe information
on data lines as number he/she needs to assign integer
bits numbers to lines, and that is absolutely arbitrary
process  In one choose one way to assigned those
bits to lines and another choose reverse way, they will
talk about completely different numbers for the same
signals on the bus. Such data lines enumeration has
no reflection on how bus actually works. And I don't
even see why it should be described just as single
integer, for example one can describe information on
data lines as set of 4 byte value, nothing wrong with
such description.

> How the device at the far end of the bus chooses
> to respond to 32 bit accesses to address X versus
> 8 bit accesses to addresses X through X+3 is entirely
> its own business and unrelated to the CPU. (It would
> be perfectly possible to have a device which when
> you read from address X as 32 bits returned 0x12345678,
> when you read from address X as 16 bits returned
> 0x9abc, returned 0x42 for an 8 bit read from X+1,
> and so on. Having byte reads from X..X+3 return
> values corresponding to parts of the 32 bit access
> is purely a convention.)

I don't follow above, one may have one read from
device address X as 32 bits returned 0x12345678,
and another read from the same address X as 32 bit
returned 0xabcdef123, so what? Maybe real example
would help.

>> Note nowhere in my above description I've talked
>> about endianity of anything: device, access (E bit),
>> KVM host, guest, hypervisor. All these endianities
>> are irrelevant to mmio interface.
>
> As soon as you try to think of the mmio.data as a set
> of bytes then you have to specify some endianness to
> the data, so that both sides (kernel and userspace)
> know how to reconstruct the actual data value from the
> array of bytes.

What actual value? In what sense? You need to bring
into discussion semantic of this h/w address to really tell
that. Driver that reads/writes is aware about semantics
of those addresses. For example devices gives chanel1 byte
value at phys_addr, gives chanel2 byte value at
phys_addr + 1, so 16bit integer read from phys_addr
will bring two chanel values into register not one 16bit
integer.

>>> Furthermore, device endianness is entirely irrelevant
>>> for deciding the properties of mmio.data[], because the
>>> thing we're modelling here is essentially the CPU->bus
>>> interface. In real hardware, the properties of individual
>>> devices on the bus are irrelevant to how the CPU's
>>> interface to the bus behaves, and similarly here the
>>> properties of emulated devices don't affect how KVM's
>>> interface to QEMU userspace needs to work.
>>
>> As far as mmio interface concerned I claim that any
>> endianity is irrelevant here. I am utterly lost about
>> endianity of what you care about?
>
> I care about knowing which end of mmio.data is the
> least significant byte, obviously.

LSB of what? Memory semantics does not have
notion of LSB. It comes only when one start
interpreting memory content. memcpy does not
have any LSB involved just bytes.

>> Consider
>> the following ARM code snippets:
>>
>> setend le
>> mov r1, #0x04030201
>> str r1, [r0]
>>
>> and
>>
>> setend be
>> mov r1, #0x01020304
>> str r1, [r0]
>>
>> when above snippets are executed memory bus
>> sees absolutely the same thing, can you tell by
>> looking at this memory transaction what endianity
>> is it? And endianity of what? I can't.
>
> That is correct. That is because the value sent out on
> the bus from the CPU is always the same: it says
> "32 bit transaction, value 0x04030201, address $whatever".

Above is just *your* choice to describe signals on
actual bus lines. I can describe the same transaction
as "put set of 4 bytes {0x01, 0x02, 0x03, 0x04} at
address $whatever". It does not change what lines
values would be during this memory transaction.

BTW could you please propose how will you see such
"32 bit transaction, value 0x04030201, address $whatever".
on ARM LE CPU in mmio.data?

If it would be {0x01, 0x02, 0x03, 0x4} it is fine with
me. That is current case ARM LE case when above
snippets would be executed by guest.

Would we  agree that the same arrangement would be
true for all other cases on ARM regardless of all other
endianities of qemu, KVM host, guest, hypervisor, etc?
If we agree on that I think we are talking about the
same thing just in different concepts. My memcpy
semantics of data.mmio[] matches those values just
fine.

>> The only thing you can tell by looking at this bus
>> memory transaction is that 0x01 byte value goes at
>> r0 address, 0x02 byte value goes at r0 + 1 address,
>> etc.
>
> No, this part is absolutely wrong, see above.

I don't see why you so attached to desire to describe
data part of memory transaction as just one of int
types. If we are talking about bunch of hypothetical
cases imagine such bus that allow transaction with
size of 6 bytes. How do you describe such data in
your ints speak? What endianity you can assign to
sequence of 6 bytes? While note that description of
such transaction as set of 6 byte values at address
$whatever makes perfect sense.

Thanks,
Victor

> thanks
> -- PMM