Re: Degenerate file access patterns

[David Kastrup]

David Kastrup <address@hidden> writes:

> Trevor <address@hidden> writes:
>
>> I'm trying to run LilyPond in Google Cloud Functions
>> <https://cloud.google.com/functions/>, and execution is ridiculously slow
>> (like 40 seconds compilation vs. 2 seconds on my laptop). A Google Cloud
>> engineer tested it and reported the following:
>>
>> "The culprit is that the lilypond binary has a bit sub-optimal file
>> access pattern (opening the same file thousands of times and reading
>> it byte by byte, causing a syscall flood - nearly 500K lseek and read
>> operations). On a local machine, because of this issue, it will spend
>> about 1s in I/O syscalls, which is half of the total execution
>> time. This currently does not play nice with our systems, getting it
>> from 1s to over half a minute."
>
> I suspect this is rather related to memory mapping files (either for
> storage management or for input).
>
> So it would be good to know just what files we are talking about here.

Take that back: one would not see syscalls for memory mapping.

It seems that ly:load uses primitive-load-path (from the Guile core) and
its implementation alternates between seeks and reads of 4K size.  That
does not seem too bad.

-- 
David Kastrup