Re: Slow history load with some values of HISTSIZE

[Casey Johnson]

Thanks! I'm happy to help.
________________________________
From: alex xmb sw ratchev <fxmbsw7@gmail.com>
Sent: Friday, February 9, 2024 9:13 PM
To: Chester Ramey <chet.ramey@case.edu>
Cc: Casey Johnson <strykre@hotmail.com>; bug-bash <bug-bash@gnu.org>
Subject: Re: Slow history load with some values of HISTSIZE



On Sat, Feb 10, 2024, 03:06 Chet Ramey 
<chet.ramey@case.edu<mailto:chet.ramey@case.edu>> wrote:
On 2/3/24 3:29 PM, Casey Johnson wrote:

> Bash Version: 5.1
> Patch Level: 16
> Release Status: release
>
> Description:
>
> The current implementation of the command history takes a long time to load
> a large HISTFILE when history is stifled.  This is because there is a
> memmove() for every line of HISTFILE once HISTSIZE lines have been loaded.
> If N is the number of lines in HISTFILE then the cost of the memmoves ends
> up being something like (N - HISTSIZE) * HISTSIZE, peaking when HISTSIZE is
> roughly N/2.

Thanks for the report and the patch. I applied it with some tweaks and
saw a dramatic performance improvement for very large history files (I
tested with one that was about 690000 entries) where HISTSIZE is
significantly smaller (I tested with HISTSIZE=100000): from around 11
seconds to less than 0.3 seconds.

++ speed optimization

greets ++

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu<mailto:chet@case.edu>    
http://tiswww.cwru.edu/~chet/