Re: [Axiom-developer] RE: algebra Makefiles with explicit depende ncies, bootstrap, fixed-points etc.

[root]

> > Next we see the transition to compiling the non-BOOTSTRAP algebra.
> > This is done in the interpsys image. It is also done in an image 
> > that has access to only the BOOTSTRAP algebra code. Thus the 
> > new compiler might not have all of the algebra-defined replacements
> > available at compile time. Notice the autoload files that contain
> > the optimizations.
> >
> 
> What do you mean by "algebra-defined replacements". Where are
> these replacements defined? How are they different than the
> "optimizations" contained in the autoload files?

The BOOTSTRAP files contain lsp and spad code. The build steps
for the initial BOOTSTRAP layer use "depsys" to compile the lisp
code. So, in this case, we see VECTOR.lsp being compiled in the
depsys image. VECTOR.lsp is the last algebra file that contains
BOOTSTRAP code. So in this phase we see:

VECTOR.lsp (BOOTSTRAP) -> VECTOR.o

Once we leave the phase (shown in the printout) we start using
an "interpsys" image to compile spad code. Much later in the console
VECTOR.spad will be compiled and replaces the lisp. However the
"interpsys" environment in VECTOR.spad differs significantly from
the "depsys" environment that compiles VECTOR.lsp BOOSTRAP code. 
So we see

VECTOR.spad + SINT.NRLIB -> VECTOR.lsp -> VECTOR.o (replaced previous version)

Suppose VECTOR.spad used something from SINT.spad. When the
VECTOR.lsp BOOTSTRAP file is compiled by depsys it contains
(+ ... ...) and generates untyped and unoptimized code.

Later when VECTOR.spad is compiled and SINT.o is available 
then the spad compiler will use the optimization information
to generate (QSADD1 ... ...)

> What do you mean by "expose". Why does this cause interpsys to
> be re-built? That is necessary if code is going to be generated
> based on new optimizations isn't it?

expose == make available
Due to my lack of foresight in choosing the VECTOR.lsp BOOTSTRAP
code I did not use the most highly optimized version. But in the
algebra recompile of VECTOR.spad later in the build this gets
corrected because the operations are made available (exposed) by
previously compiled .spad code. So:

VECTOR.lsp (BOOTSTRAP) uses SINT.o

But the VECTOR.lsp has (+ ... ...) instead of (QSADD1 ... ...)
because the rewriting of "+" to "QSADD1" is done by the spad
compiler, not the GCL compiler and we are not yet in a position
to use the spad compiler.

Later when 

VECTOR.spad -> VECTOR.lsp -> VECTOR.o (replaced previous version)

the spad compiler was invoked and it gets a chance to change
"+" to "QSADD1". 

Because I grabbed the wrong version of .lsp file for BOOTSTRAP
code you are able to see this difference. The work you and steve
are doing is to find stable versions of the .lsp files.

> > Since these are only optimizations the final results should not
> > be affected.
> >
> 
> But it could have an impact on overall performance, right?

Oh, clearly. (QSADD1 ... ...) is faster than (+ ... ...)

Furthermore, there is another optimization step we could take
during the build but we do not (yet). When the GCL compiler
compiles VECTOR.lsp it generates VECTOR.fn. If you look in these
.fn files you'll find exact type information for each function call.

VECTOR.lsp -> VECTOR.o + VECTOR.fn (unoptimized, but type generated)

If this exact type information was available at compile time then
the GCL compiler can lay down faster code. However there is a 
bootstrap problem here also since the GCL compiler has to compile
the code one time in order to generate the .fn file.

VECTOR.lsp + VECTOR.fn -> VECTOR.o (type optimized)

So the full, proper procedure, which I have yet to do, would actually
call the GCL compile once to generate the .fn file with exact type
information for function calls, load the .fn file, and the recompile
the lisp file. Since the second compile has exact type information 
the generated code will be (potentially much) shorter.

For each lisp file do
(
  VECTOR.lsp -> VECTOR.o + VECTOR.fn (unoptimized, but type generated)
  VECTOR.lsp + VECTOR.fn -> VECTOR.o (type optimized)
)


> Still after reading the interp and algebra Makefiles again,
> I do not see why interpsys is being rebuilt after I delete
> all the *.NRLIB's and repeat the 'make'. I am still missing
> something somewhere. Is there some "missing link" that still
> connects the algebra/Makefile back to the inter/Makefile?

I have to think about this (I'm getting ready for work so I'll
give it some "drive-time cycles") but I believe that the issue
is related to using fresh databases. The databases are used 
during the algebra compiles to resolve algebra calls. However
recompiling the algebra generates NRLIBs and, from these NRLIBs 
a fresh database is built thus:

DATABASES -> NRLIBs -> FRESH DATABASES

This is yet a third bootstrap issue. There are more but not with
the algebra files.

Tim