Re: Surely 'bzr update' shouldn't be this slow?

[Alan Mackenzie]

Hi, Óscar,

On Thu, Jan 07, 2010 at 02:56:02PM +0100, Óscar Fuentes wrote:
> Alan Mackenzie <address@hidden> writes:

> [snip]

> > I beg your pardon?  My machine is a 1.2 GHz Athlon.  That is NOT a
> > slow machine by any measure, except that even faster machines are now
> > common.

> I think that that is the definition of "slow machine" :-)

It's a machine which is more than fast enough for every aspect of Emacs
development other than running bzr.

> > Why does Bazaar need to "process" this data?  It's essentially doing
> > copying, with some accompanying administrivia.

> It is not copying, and in that case it can't simply copy the data. A
> shared repository contains data from all the branches it encloses. When
> you branch from it, only the data that belongs to the branch you want
> must be transferred.

It IS copying, conceptually - the content of the branched repository
contains a portion of the original branch, and nothing else.  So it has
to do a bit of filtering on the files' logs.  There are ~2000 files in
Emacs, and that copying/filtering took 39 minutes, about 2300 seconds.
It's taking over a second per file (at ~100% CPU usage).  A second is
about how long my PC takes to compile each C file in .../emacs/src.

> I don't know why Bazaar does not copy the files under .bzr when you
> clone a branch when it is on the same format that the target,
> though. A good question for it ml.

> > Is it doing heavy number crunching in Python, when it really needs a C
> > module, or something like that?

> The inner loops are written in C.

> > I just did a 'bzr update' on my .../trunk.  It took 23 minutes,
> > transferring nearly 200Mb to/from savannah in the process.  This
> > compares with all our source files (.c, .h, .el) being ~64Mb.  Could
> > it be that 'bzr update' just downloads the whole repository again?
> > Or has somebody else raised this issue on another thread that I've
> > missed?

> Yup, that was recently discussed here. It was an exceptional case (I
> hope).

There have been, perhaps, ~100 files updated since I first downloaded the
repository last Friday.  For each changed file, bzr transferred ~2Mb
between my PC and savannah.  Why?  This is ludicrous.

Hopefully it was an exceptional case, but I'd not changed my .../trunk at
all since downloading it, so anything exceptional was at the savannah
end.

I'm about to fix a bug which will involve ~100 bytes change to a C file
and ~200 bytes log message and ChangeLog addition.  How much will the bzr
commit operation transfer?  Hopefully, several kilobytes, no more.

Any chance you could point out that other thread to me?

> > There seems to be a substantial mismatch between the assumptions of
> > the bzr project and the realities of the Emacs project.  My
> > impression is that bzr is so slow as to be barely usable at the
> > moment.

> Apart from the 200MB download, do you think that bzr is too slow on your
> daily Emacs work? Which operations are too slow for you?

Yes, bzr is too slow for me.  My first checkout took, perhaps, an hour
and a half, but I can cope with that.  'bzr branch' (to a random place)
took 40 minutes.  'bzr branch' to the Right Place took a few seconds, and
this is the only bzr thing I've done so far which has been reasonable.
'bzr update' took 23 minutes, and this is the killer, the operation which
will make Emacs development such a frustrating, miserable experience; on
CVS, it would have been faster on my 33MHz 486 with 33kbaud modem.

> Sometimes I work on a netbook, which maybe is comparable to your 1.2GHz
> Athlon on CPU power, and some operations are slow indeed (>20 seconds
> for the log of a file.) I raised this on the Bazaar ml, but most people
> there think that Bazaar is fast enough and no extensive work is planned
> for performance improvements.

Their basic assumptions don't match the Emacs project, for whatever
reason.

> -- 
> Óscar

-- 
Alan Mackenzie (Nuremberg, Germany).