[Monotone-devel] Re: why is monotone so slow?

[Georg-W. Koltermann]

Hi Graydon,

Am Di, den 14.09.2004 schrieb graydon hoare um 16:16:
...
> on monotone:
> 
>    - calculate the working copy state
>    - calculate the set of branch certs you trust, out of all the
>      revisions in the database (nb. you have now read a sizeable
>      portion of the database)
>    - look over the graph of all those revisions and discard
>      ancestry certs you don't trust
>    - compute the set of heads from that set (discard all entries
>      with live children)
>    - check to see if there is a unique head equal to the current
>      working copy state

I suspect that does not bite me, because:

        hunter[42]$ time monotone heads
        monotone: fetching heads of branch 'r5-features'
        branch 'r5-features' is currently merged:
        1163bb07e6261605ba98bc033166d64c8aa582be address@hidden 
2004-09-14T10:02:00
         
        real    0m0.053s
        user    0m0.038s
        sys     0m0.008s
        hunter[43]$

I only imported my source code once and committed about 5 changes over
the last week, all in linear sequence, that explains how cheap it is in
my case.

> some of this is avoidable: after the changeset branch is complete
> the graph can have its heads calculated much faster, for example,
> and without reading the whole cert table. some of it is not as
> avoidable: monotone doesn't have a single concept of up-to-date,
> it's in the eyes of the client and its trust function, so there
> is always a possibility for slowdown there.
> 
> however, all this is nonsense speculation until you get out a
> profiler. I have profiled monotone many times in the past and
> will continue to do so any time I'm working on speed issues;
> doing otherwise is a waste of time. typical hotspots I've run
> into before:
> 
>    - gzip longest sequence scanning (tunable by adjusting
>      which in-memory representations remain compressed and
>      which do not, and what thresholds are used)
>    - SHA1 calculation (often tunable by caching IDs sensibly)
>    - sqlite statement parsing (tunable by move to sqlite3
>      and use of prepared statements)
>    - sqlite execution (tunable by adjustment of page size,
>      addition of indices, move to sqlite 3, etc.)
>    - RSA inner loop (often tunable by caching trust values
>      or rearranging logic)
>    - bad i/o buffering strategies (tunable)
> 
> so, "yes", it can usually be made lots faster. I can't guess
> off hand what the hotspot is in your email, but it can probably
> be made to go away.

I should have been more precise in my original question: Do you perhaps
have a pointer to how I can tune sqlite?  Or a pointer to how I could
replace it with sqlite3?  I am currently just playing, it would be no
problem if you tell me to install another branch of monotone (which
might have those improvements) and start importing my source afresh.

--
Thanks,
Georg.