Re: wc -l AVX code 10%+10% speedup

[Pádraig Brady]

On 30/03/2024 14:52, Evgeny Nizhibitsky wrote:
> Dear GNU coreutils maintainers,
>
> It seems that I found a way to both speed-up (~10%) and simplify (13
> insertions, 43 deletions) the wc -l avx code while playing with it, at
> least on several million to 1 billion row files I tested with my cpu.
>
> It mostly involves using _mm256_movemask_epi8 and __builtin_popcount
> instead of the two accumulators handling that allowed me to increase the
> buffer size.
>
> I also have a further ~10% improvement in code by using 2 separate threads
> instead of 1 to mitigate the usr time overhead, although it’s naturally
> more complicated.
>
> Whom should I discuss this potential contribution with?

You can propose the code here.
The AVX adjustments sound very interesting.

The threading sounds a bit less useful TBH,
and might introduce more complexity/overhead
especially in the common case.
BTW threaded line counting was discussed at:
https://www.pixelbeat.org/docs/unix-parallel-tools.html

thanks,
Pádraig