Re: Slow history load with some values of HISTSIZE

[Chet Ramey]

On 2/3/24 3:29 PM, Casey Johnson wrote:

> Bash Version: 5.1
> Patch Level: 16
> Release Status: release
>
> Description:
>
>       The current implementation of the command history takes a long time to 
> load
>       a large HISTFILE when history is stifled.  This is because there is a
>       memmove() for every line of HISTFILE once HISTSIZE lines have been loaded.
>       If N is the number of lines in HISTFILE then the cost of the memmoves ends
>       up being something like (N - HISTSIZE) * HISTSIZE, peaking when HISTSIZE 
> is
>       roughly N/2.

Thanks for the report and the patch. I applied it with some tweaks and
saw a dramatic performance improvement for very large history files (I
tested with one that was about 690000 entries) where HISTSIZE is
significantly smaller (I tested with HISTSIZE=100000): from around 11
seconds to less than 0.3 seconds.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

OpenPGP_signature.asc
Description: OpenPGP digital signature