[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Compilation speed
From: |
Lars Ingebrigtsen |
Subject: |
Compilation speed |
Date: |
Thu, 06 Aug 2020 17:20:13 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) |
A nice thing about new, shiny machines with a bunch of cores is that it
makes saying "make bootstrap" faster when I want to check stuff.
However, there's a couple of sections that are single-threaded, which
makes the machine mostly idle while building, and I wondered whether
anybody has looked into these things...
I did a bootstrap (with make -j8) and timestamped the lines to see
whether anything in particular stands out.
[larsi@stories ~/src/emacs/trunk]$ emake bootstrap 2>&1 | ts -s
00:00:00 rm -f libXMenu11.a ./*.o deps/*
...
First deleting a lot of stuff... that's fine...
00:00:03 running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create
--no-recursion
00:00:03 checking for xcrun... no
00:00:03 checking for GNU Make... make
[...]
00:00:22 config.status: executing etc-refcards-emacsver.tex commands
And then ./configure takes 19 seconds, and that's fully single-threaded,
I believe? And... I'm guessing there's no way to get that to be
multi-threaded?
But I've also long wondered whether anybody has actually pruned the
tests lately? Are all those tests (for things that seem really trivial,
mostly) really necessary these days?
And then compilation of Emacs and the manual starts, and that bit uses
all eight cored:
00:00:22 GEN alloca.h
[...]
00:00:22 GEN ../../info/eintr.info
00:00:22 GEN sys/random.h
00:00:22 GEN sys/select.h
00:00:22 GEN ../../info/elisp.info
00:00:38 GEN sys/stat.h
00:00:38 GEN sys/time.h
00:00:38 GEN sys/types.h
[...]
00:00:50 CC minibuLoading loadup.el (source)...
00:00:50 dump mode: pbootstrap
So the build of the temacs binary just takes 28 seconds.
00:00:50 Loading emacs-lisp/byte-run (source)...
00:00:50 Loading emacs-lisp/backquote (source)...
[...]
00:01:02 Dump complete
And dumping takes 12 seconds. Then some other bits are compiled and
then we start the pbootstrap:
00:01:07 ./temacs --batch -l loadup --temacs=pbootstrap
This seems to proceed in parallell until we reach:
00:01:17 INFO Scraping files for loaddefs.el...
[...]
00:01:54 INFO Scraping files for loaddefs.el...done
Which is single-threaded and takes a whopping 37 seconds! Could that
possibly be parallelised in some way? Probably be tricky...
00:01:54 Loading loadup.el (source)...
[...]
00:01:58 Dump complete
Another dump; takes just four seconds.
00:02:00 ELC ../lisp/language/khmer.elc
00:02:00 ELC ../lisp/language/korean.elc
[...]
00:02:46 ELC progmodes/cc-styles.elc
00:02:46 ELC progmodes/cc-mode.elc
And finally, all the remaining (undumped) .el files are compiled, and
this uses all the cores, so it takes just 46 seconds.
Hm... well, I guess if somebody wanted to speed up compilation, then
the loaddefs scraping is the biggest single item here to look at.
But to sum up, (+ 22 12 37 4) => 75 seconds of the Emacs compilation is
single-threaded, while 90 seconds are multi-threaded (and as machines
get more and more cores, we'll probably see the single-threaded parts
take more than 50% of the time spent).
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no