Re: [Monotone-devel] speed of 0.99

[Timothy Brownawell]

On 11/27/2010 10:55 PM, Timothy Brownawell wrote:
> On 11/19/2010 04:31 AM, Markus Wanner wrote:
>> Hi,
>>
>> I've recently upgraded to monotone 0.99, mainly to get the new commit
>> message editor. Thanks for that!
>>
>> However, it seems to me it's quite a bit slower than previous versions.
>> Which weren't exactly fast, either, but 'mtn status' and 'mtn ls
>> branches' seems to take half an eternity. (Admittedly, I'm keeping all
>> of my projects in one database file, which certainly doesn't help speed,
>> because that one file is getting close to 1 GiB in size).
>>
>> Is this just subjective, or did anybody else notice a similar slowdown?
>> It this worth investigating?
> 
> My db is 785MB, and the copies for 0.45 and 0.48 are 639MB. This is in a 
> checkout of the monotone tree.
> 
> | hot cache | cold cache |
> | head | 0.45 | 0.48 | head | 0.45 | 0.48 |
> status | .16s | .16s | .16s | 1.2s | .45s | 1.2s |
> w/o inodeprints | .22s | .22s | .22s | 1.7s | .89s | 1.5s |
> ls branches | .13s | 4.1s | .13s | 1.1s | 5.3s | .97s |
> 
> So it looks like 'ls branches' may have slowed down a tiny bit since 
> 0.48, and 'status' is slightly slower since 0.45 (but only on a cold 
> cache).
> 
> I'd think that the additional time in 'status' is from the extra info we 
> output now, probably mostly the check for whether a commit would create 
> an extra head. But 'mtn heads' only takes .15s cold-cache, so there's 
> still around .7s missing...

The extra time for status was actually in the "will create a new branch"
check, due to not-so-great indexing on the revision_certs table. With the
below change, 'mtn status' on a cold cache takes .41s / .77s with/without
inodeprints (and .13s / .19s with/without inodeprints on a hot cache). So
actually a bit faster than 0.45 was. This also speeds up 'ls branches' to
.75s cold-cache.

...if I put the db and workspace on a spinning-oxide drive, I get these
cold-cache times:

                  | fixed | 0.45 | 0.48
status            | 1.1s  | 1.0s | 11s
  w/o inodeprints | 6.4s  | 6.4s | 17s
ls branches       | 2.8s  | 18s  | 11s


Do we mind having an index-only schema change between 0.99 and 1.0? It
sounds like we'd get a noticeably better user experience...



----------------------------------------------------------------------
Revision: f94d3092bdb3a65bceb1acd654d3e3c3f7df2676
Parent:   81b1f0ed293ff2b8a0c0329fc400a2809ebb37d5
Author:   Timothy Brownawell  <address@hidden>
Date:     11/28/2010 12:17:39 PM
Branch:   net.venge.monotone.revision_certs_indexing

Changelog: 

Better indexing for the revision_certs table, so that looking up the
certs on a particular revision is faster (can pull everything from the
index, without having to go look up in the table).

Changes against parent 81b1f0ed293ff2b8a0c0329fc400a2809ebb37d5

  added    
tests/schema_migration/c3a13c60edc432f9a7739f8a260565d77112c86e.mtn.dumped
  patched  migrate_schema.cc
  patched  schema.sql
  patched  tests/schema_migration/__driver__.lua

============================================================
--- schema.sql  e95592782a6420be6e3a0e20f8d053147de38d70
+++ schema.sql  c1f635361a6cae78c93018b4ac3d02546c8dc0c8
@@ -107,7 +107,8 @@ CREATE TABLE revision_certs
        unique(name, value, revision_id, keypair_id, signature)
        );
 
-CREATE INDEX revision_certs__revision_id ON revision_certs (revision_id);
+CREATE INDEX revision_certs__revnameval ON revision_certs (revision_id,
+       name, value, keypair_id, signature);
 
 CREATE TABLE branch_epochs
        (
============================================================
--- migrate_schema.cc   be2b52eb8a339ff92e248ae9cbb506eaecd22ffe
+++ migrate_schema.cc   d6161bf90f49f20afd61716ce2ab56da58567bf4
@@ -767,6 +767,11 @@ char const migrate_certs_to_key_hash[] =
   "DROP TABLE revision_certs_tmp;"
   ;
 
+char const migrate_better_cert_indexing[] =
+  "DROP INDEX revision_certs__revision_id;\n"
+  "CREATE INDEX revision_certs__revnameval ON revision_certs (revision_id,\n"
+  "       name, value, keypair_id, signature);";
+
 namespace {
   struct branch_leaf_finder_info
   {
@@ -962,9 +967,12 @@ const migration_event migration_events[]
   { "0c956abae3e52522e4e0b7c5cbe7868f5047153e",
     migrate_add_file_sizes, 0, upgrade_regen_caches, regen_file_sizes },
 
+  { "1f60cec1b0f6c8c095dc6d0ffeff2bd0af971ce1",
+    migrate_better_cert_indexing, 0, upgrade_none, regen_none },
+
   // The last entry in this table should always be the current
   // schema ID, with 0 for the migrators.
-  { "1f60cec1b0f6c8c095dc6d0ffeff2bd0af971ce1", 0, 0, upgrade_none, regen_none 
}
+  { "c3a13c60edc432f9a7739f8a260565d77112c86e", 0, 0, upgrade_none, regen_none 
}
 };
 const size_t n_migration_events = (sizeof migration_events
                                    / sizeof migration_events[0]);



-- 
Timothy

Free public monotone hosting: http://mtn-host.prjek.net