Re: Suggestions for TCG performance improvements

[Vasilev Oleg]

On 12/3/2021 8:32 PM, Alex Bennée wrote:
> Vasilev Oleg <vasilev.oleg@huawei.com> writes:
>
>> On 12/2/2021 7:02 PM, Alex Bennée wrote:
>>
>>> Vasilev Oleg <vasilev.oleg@huawei.com> writes:
...skipped...
>>> I did ponder a debug mode which would keep the last N tables dropped by
>>> tlb_mmu_resize_locked and then measure the differences in the entries
>>> before submitting the free to an rcu tasks.
>>>> The mentioned paper[4] also describes other possible improvements.
>>>> Some of those are already implemented (such as victim TLB and dynamic
>>>> size for TLB), but others are not (e.g. TLB lookup uninlining and
>>>> set-associative TLB layer). Do you think those improvements
>>>> worth trying?
>>> Anything is worth trying but you would need hard numbers. Also its all
>>> too easy to target micro benchmarks which might not show much difference
>>> in real world use. 
>> The  mentioned paper presents some benchmarking, e. g. linux kernel
>> compilation and some other stuff. Do you think those shouldn't be
>> trusted?
> No they are good. To be honest it's the context switches that get you.
> Look at "info jit" between a normal distro and a initramfs shell. Places
> where the kernel is switching between multiple maps means a churn of TLB
> data.
>
> See my other post with a match of "msr ttrb"
Sorry, couldn't find what you are referring to. Could you, please, share
a link?
>>>> Another idea for decreasing occurence of TLB refills is to make TBs key
>>>> in htable independent of physical address. I assume it is only needed
>>>> to distinguish different processes where VAs can be the same.
>>>> Is that assumption correct?
>> This one, what do you think? Can we replace physical address as part
>> of a key in TB htable with some sort of address space identifier?
> Hmm maybe - so a change in ASID wouldn't need a total flush?

No, I think it would need a flush since regular memory accesses need to
be in the correct address space. But, we won't need to access TLB when
looking for the next TB. Also, TLB wouldn't need to be filled with code
pages, only data pages.

Overall, thanks for your feedback on those ideas.

Oleg


...skipped...