Re: Finalizing 'inhibit-automatic-native-compilation'

[Lynn Winebarger]

On Sun, Feb 5, 2023 at 2:40 AM Eli Zaretskii <eliz@gnu.org> wrote:
> I asked because Emacs 29 adds some changes specifically intended to
> solve the problem of infinite recursion in trampoline compilation.
I'll test again after I reimplement the dumping support for text
properties, but it might be a while.
> > As far as I know, emacs still doesn't support dumping arbitrary
> > native-compiled libraries at compile-time.
>
> What problems do you see if you try?
   I filed a number of bug reports for tracking purposes at the time.
1.  Lack of support for purifying certain object types, including but
not limited to text properties
2.  Keymap definitions that were incompatible with purification
3.  The strict requirements around autoload and load order in
compile-time dumping - plenty of libraries just aren't factored to be
loaded without autoloads, or weren't at the time of the experiment
4.   Issues with variable initialization and code evaluated at
load-time rather than at init, as noted at the end of my previous
reply
5   There were some issues with docstrings, but those should have been
resolved the move to always load those from .elc files.
6.  Spawning the asynchronous compile process with whatever dump file
the session was loaded with is asking for trouble.  It would be better
to specify a "known good" dump file, like one limited to loadup with
no site files, that has been tested.  The compiler process dump file
could be a customization variable if tweaks are required.

Presumably 1 & 2 will be resolved by removing purification, whenever
that happens.  Number 3 is fairly easy to resolve by introducing a
variable to control when that strict ordering is required, so it can
be turned off after the explicit loading is done in "loadup".  As
Stefan Monnier suggested, the site-load phase is probably best done as
a third build phase, following the bootstrap and loadup-only dumps.

> > Which is too bad, because the dumped version is a LOT faster to
> > startup.
>
> Are you sure? is this with native-compilation or without it?

Very sure.  I had dumped emacs with 4000-5000 loaded libraries, all
precompiled .eln files, and it would pop up in 5 seconds or less the
first time it loaded. Subsequent starts (while the first was still
running) appeared to load even faster,  I assumed because the eln
files are loaded as shared objects and shared between processes by the
kernel.

> I don't see why it would be faster when loading a dumped Emacs: the
> pdumper file records only the names of the *.eln files, not their
> contents, so Emacs still has to load all of the *.eln files recorded
> in the pdumper file, and that takes time and doesn't change much when
> the names are recorded in emacs.pdmp.

I need to do proper testing, but part of the speedup is probably due
to being able to skip checking whether the source file has the right
checksum.  I recently submitted a bug report showing that when I had
900+ packages installed and native-compilation enabled for 30.0.50,
that startup time is dominated by locate-file, presumably because the
eln loader has to find the right source file to check.  Otherwise, the
length of the load-path would have minimal effect, due to the eln
caching.

> If you want to dump more libraries than Emacs does by default, why not
> modify loadup.el to load those libraries at build time?  Then they
> will be dumped together with all the rest, and the loading won't need
> any changes on your side.

See above.  Also, I would use site-load to avoid polluting loadup.el.
The biggest advantage of dump-emacs-portable is that it doesn't have
the prohibition against autoloads and circular requires that dump-mode
does, or the problems with purification.
I'm happy to submit patches once I've been able to set up the testing
infrastructure necessary to be sure they're sound.  And actually done
the work, which is non-trivial.

Lynn