Re: [Monotone-devel] Re: network costs of history copying

[Nathaniel Smith]

On Wed, Jan 18, 2006 at 09:35:57AM +0300, Petr Ovtchenkov wrote:
> Hello Nathaniel,
> 
> > I just did a pull of net.venge.monotone alone, to test.  (Some of
> > the contrib branches contain some rather large stuff, like a copy of a
> > bunch of big 3rd-party java libraries, so I left those out to get a
> > fair test).  Netsync transferred 25.6 megabytes.
> 
> What branches you mean? The last time I have made initial pull (Oct 2005), I 
> saw

net.venge.monotone.contrib and net.venge.monotone.contrib.monotree in
particular.  You could use
  monotone pull venge.net "net.venge.monotone*" --exclude 
"net.venge.monotone.contrib*"
if you prefer.

I don't actually have the current statistics for net.venge.monotone*.
The netsync rewrite may have reduced the bytes in somewhat (it
certainly reduced the "bytes out" to essentially nothing on fresh
pulls).

> following:
[...]
> monotone: bytes in | bytes out | certs in | revs in | revs written
> monotone:    42.7M |      1.9M |    18210 |    3932 |         3932
> monotone: successful exchange with venge.net
> 8609.125u 856.100s 2:39:33.01 98.8%     0+0k 0+0io 1pf+0w
> 
> Please, correct me, because 26M is acceptable for me, but 50M in a 2h40m 
> isn't.

The previous post was about the amount of data transferred, rather
than the time it took to process.  They're somewhat separate -- in
particular, pure optimization will not reduce the amount of data
transferred much, we should plan for that to always be there modulo
addition of new ways of working with your database like the "partial
pull" I discussed.

The time is a different matter in that it should be fixable, and we're
working on fixing it.  Now that rosters are in, we seem to be bound on
the amount of time we spend fiddling with the database and writing
stuff into it -- so we just need to make that faster.  If anyone's
interested in helping with this, we have some ideas for what to do...

The end goal is for netsync to be network bound -- it should pull as
fast as your internet connection lets it.

-- Nathaniel

-- 
In mathematics, it's not enough to read the words
you have to hear the music