Re: [Axiom-developer] Bootstrap documentation.

[Martin Rubey]

Dear Waldek,

Waldek Hebisch <address@hidden> writes:

> I want to include some documentation of current algebra bootstrap used
> in wh-sandbox.  Below is what I have now.  Any comment (or things that
> need better explanation)?

This is GREAT!

I just read your overview, and believe that I understood.  I threw in
some articles and spelled out OTOH and ATM, since when I first read them
they gave me a headache.  Furthermore, I removed some parentheses, since
I dislike them.  I tried not to change linebreaks, so emacs ediff or
similar should make cherry picking easy.

I'm not English native, so please correct me if where I'm wrong.

I guess the whole bootstrap problem would go away with "extend"...  What
a pity!

Martin

\section{The bootstrap problem\dots}

The current Axiom compiler compiles each constructor (category, domain
or package) separately.  In order to perform type checking Axiom
needs type information from other constructors.  To collect
type information about a constructor Axiom must compile it.
This creates a bootstrap problem: in order to compile a constructor
one must first compile many other constructors.

In principle the bootstrap problem could be avoided by organizing algebra
into layers, so that layer $0$ is independent of other layers and
each higher layer depends only on lower layers.  However, the current
Axiom algebra contains a lot of cyclic dependencies: for example, a substantial
part of algebra depends on [[Integer]], but [[Integer]] has a rich structure
and depends on many categories which in turn depend on [[Integer]].

\section{\dots and its solution}

At the moment this bootstrap problem has no proper resolution: Axiom
avoids it simply using type information collected in the past
and stored in databases [[interp.daase]] and [[category.daase]].
This solution means that any attempt to change the signature of an exported
operation is likely to cause build failure.

Using the type information stored in databases solves only part of
the bootstrap problem: internally, Axiom types are represented by
executable code and sometimes Axiom needs to load actual compiled
code.  The exact rules telling when Axiom needs compiled code
and when it is enough to have info in databases are unknown.
Axiom needs to load categories given as argument to [[Join]],
because [[Join]] is implemented as a runtime operation.  It
seems that Axiom also needs to load a category given as an argument
to a contructor if this category itself has arguments.

During normal compilation Axiom tries to optimize (i.e., expand inline)
calls to operations from core domains, which are listed in the
[$optimizableConstructorNames]] variable.  To do this, Axiom
has to load these domains.  In the past, this optimization required
keeping compiled Lisp corresponding to core domains.  In fact, what
% I don't understand the following half-sentence. Furthermore, it is not
% clear to me what is past and what cis current behaviour.
was stored was a cut breaking cycles involving core domains.
Currently Axiom has a special flag [[$bootstrapDomains]] to
disable this optimization.

Even after using the [[$bootstrapDomains]] flag there are still
cyclic load dependencies caused by constructor arguments, but
those problem are resolved using the [[$bootStrapMode]] flag,
which causes relaxed type checking and skips executable parts
of categories.

The final process looks as follows: first, categories are
topologicaly sorted with respect to [[Join]] --  recall that arguments to
[Join]] must be compiled before a category using [[Join]].
This order is slightly tweaked to avoid problems due to
constructor parameters.  In the first pass categories
are compiled using [[$bootStrapMode]] set to [[t]].  This
makes compiled code of categories available to second pass.
In the second pass categories and core domains are
compiled setting [[$bootstrapDomains]] to [[t]].  We have
to compile categories because categories may have associated
default domains which also may be loaded.  We also compile a
few domains which are needed for unknown reasons - if we
skip them, the build will fail.  In the third pass we
compile categories and core domains in normal mode.
Finally, we compile the rest of algebra in alphabetical order.
More precisely, we let make choose the order, but the list we give is
sorted alphabetically, and make processes it in order.

\section{Adding new constructors}

Adding a new constructor is different from the above process.  It is
easier because a new constructor can freely use functions provided
by standard algebra.  On the other hand, information about new constructors
is absent from Axiom databases, so cyclic dependencies between new
constructors have no easy solution.  However if one is adding
mutually dependent constructors without cyclic dependencies
the solution is easy:  compile all dependent constructors in
a single Axiom image, using apropriate order.  The [[guess-pkg]]
target ilustrates this method.