Re: Surely 'bzr update' shouldn't be this slow?

[Óscar Fuentes]

Alan Mackenzie <address@hidden> writes:

[snip]

>> > Why does Bazaar need to "process" this data?  It's essentially doing
>> > copying, with some accompanying administrivia.
>
>> It is not copying, and in that case it can't simply copy the data. A
>> shared repository contains data from all the branches it encloses. When
>> you branch from it, only the data that belongs to the branch you want
>> must be transferred.
>
> It IS copying, conceptually - the content of the branched repository
> contains a portion of the original branch, and nothing else.  So it has
> to do a bit of filtering on the files' logs.  There are ~2000 files in
> Emacs, and that copying/filtering took 39 minutes, about 2300 seconds.
> It's taking over a second per file (at ~100% CPU usage).  A second is
> about how long my PC takes to compile each C file in .../emacs/src.

Things are not so simple. Please note that bzr stores almost 100,000
revisions of Emacs development in a bit more that 200 MB. And each
revision is related to other revisions and that relations needs
traversing. It is obvious that the storage system is far from trivial,
so any complexity assumption based on concepts like files or chunks of
bytes are, very likely, wrong.

>> Yup, that was recently discussed here. It was an exceptional case (I
>> hope).
>
> There have been, perhaps, ~100 files updated since I first downloaded the
> repository last Friday.  For each changed file, bzr transferred ~2Mb
> between my PC and savannah.  Why?  This is ludicrous.
>
> Hopefully it was an exceptional case, but I'd not changed my .../trunk at
> all since downloading it, so anything exceptional was at the savannah
> end.
>
> I'm about to fix a bug which will involve ~100 bytes change to a C file
> and ~200 bytes log message and ChangeLog addition.  How much will the bzr
> commit operation transfer?  Hopefully, several kilobytes, no more.

When bzr communicates with upstream by the http or sftp protocols, it
needs to read or write lots of data because the needed information is
scattered all over the repository files.

In theory, installing the Bazaar smart server in Savannah should fix
those problems.

> Any chance you could point out that other thread to me?

The subthread starts here:

http://article.gmane.org/gmane.emacs.devel/119410

>> Apart from the 200MB download, do you think that bzr is too slow on your
>> daily Emacs work? Which operations are too slow for you?
>
> Yes, bzr is too slow for me.  My first checkout took, perhaps, an hour
> and a half, but I can cope with that.  'bzr branch' (to a random place)
> took 40 minutes.  'bzr branch' to the Right Place took a few seconds, and
> this is the only bzr thing I've done so far which has been reasonable.
> 'bzr update' took 23 minutes, and this is the killer, the operation which
> will make Emacs development such a frustrating, miserable experience; on
> CVS, it would have been faster on my 33MHz 486 with 33kbaud modem.

If the exceptionally long time that it took for certain update is the
biggest problem you found so far, there is no reason to worry. Because
it is not usual (except for that case, bzr needs less than 3 minutes for
updating on my netbook) and because if it becomes too frequent people
here will force the Savannah maintainers to install the smart server.

[snip]

-- 
Óscar