axiom-developer
[Top][All Lists]

## Re: [Axiom-developer] sbcl and Axiom

 From: root Subject: Re: [Axiom-developer] sbcl and Axiom Date: Wed, 26 Jul 2006 00:19:24 -0400

CY,

============================================================
BUILD ORDER

Axiom runs a tree of Makefiles, starting with the root Makefile.
Each subdirectory contains a Makefile.pamphlet. The system does
a recursive walk of the Makefile tree by subdirectory.

In a subdirectory running a Makefile that Makefile has the
responsibility to set up global conditions for all of the Makefiles
in each child subdirectory.
For each subdirectory
set up global conditions (eg directories)
notangle the Makefile.pamphlet ==> Makefile
( cd subdir ; make )

You can see what the build will do by recording the console
output of a system build. The exact sequence necessary to build
the system is laid out in linear form during system build. Every
make stanza outputs a line that says what it is doing. Each line
that is output has a unique number (for the subdirectory) as a
prefix to the line. So you'll see things like:

32 making /tmp/axiom49/obj/linux/graph/view2D/write2d.o from
/tmp/axiom49/int/graph/view2D/write2d.c

Notice the '32' prefix which gives a clue about which make stanza it is.
This line is output from the Makefile running in the
src/graph/view2D
subdirectory. If you look at the src/graph/view2D/Makefile.pamphlet and
search for 'echo 32' you will see the stanza:

${MIDOBJ}/write2d.o:${MIDINT}/write2d.c ${LINC}/write.h${HEADERS}
@ echo 32 making ${MIDOBJ}/write2d.o from${MIDINT}/write2d.c
@ ( cd ${MIDOBJ} ;${CC} -c ${CFLAGS}${MIDINT}/write2d.c )

This stanza says:

if you need to build  ${MIDOBJ}/write2d.o then you must first build${MIDINT}/write2d.c
${LINC}/write.h${HEADERS}
once these have been built it echos the '32' line and then
it builds ${MIDOBJ}/write2d.o ============================================================ BUILD-TIME DIRECTORY STRUCTURE There are 4 top-level directories${SRC} -- the source tree (human written)
${INT} -- the intermediate (machine independent, machine generated)${OBJ} -- the object files (machine dependent, machine generated)
${MNT} -- the final ship tree and within each Makefile combinations of these make local variables. The system is set up so that it is possible to copy the SRC and INT directory to another system and restart the build. Historically this was important because it used to take 3 weeks to build a system from scratch (well, not scratch but from a previous system). The makefiles are designed so you could mount NFS disk space on multiple systems and build them all in parallel. Axiom used to build on PCs, Symbolics, AIX, Solaris, and VM/370 all from the same SRC and INT. The target system is named by the${SYS} variable,
these days usually just linux. The target system is determined from
the $AXIOM variable so: export AXIOM=/tmp/axiom/mnt/linux ^^^^^ the target and you could build others by NFS mounting the source directory and setting the AXIOM variable such as: export AXIOM=/tmp/axiom/mnt/solaris export AXIOM=/tmp/axiom/mnt/freebsd export AXIOM=/tmp/axiom/mnt/symbolics export AXIOM=/tmp/axiom/mnt/aix etc... So the basic idea is that you start from the sources${SRC},
cache a lot of machine independent work in ${INT}, use${OBJ} as a machine specific temp area (.o files, etc),
and create a machine-specific "ship" system in ${MNT}. Once the machine-specific directory exists (${MNT}/linux, ${MNT}/aix, etc) you can throw away everything else, including the sources. ============================================================ BUILD-TIME INTERMEDIATE IMAGES We need to build lisp images for specific purposes. Initially we need a lisp image that contains our patches. That is machine-specific, machine-generated code so it logically belongs in the OBJ directory (see above). And since it is system dependent it lives under the SYS subdirectory. In this case we get${OBJ}/${SYS} == obj/linux If we look in obj/linux we see lisp bootsys depsys interpsys debugsys So the key steps that use these images amount to: step 0: build noweb step 1: build a lisp with our patches ==> lisp if we had a fully patched lisp (or did not depend on any special features) we could just copy a lisp rather than build one. step 2: build a lisp with the boot translator ==> bootsys extract the boot translator clisp files from the boot files into INT load the boot translator into the lisp image save it as bootsys step 3: translate (use bootsys) translate the boot files into clisp files using bootsys after this point bootsys is useless and we abandon the image step 4: build a lisp image with all the macros ==> depsys we use lisp macros which are needed at compile time but not runtime we load those macros into a lisp image and save it as depsys step 5: compile lisp (use depsys) we use depsys to compile the clisp/lisp/lsp files clisp is boot-generated code lisp is lisp-original code lsp is GCL generated code step 6: build a lisp image that contains the interpreter/compiler ==> interpsys we load all the compile files into lisp we can preload some algebra code for efficiency and save it as interpsys (eventually copied as AXIOMsys into${MNT}/\${SYS}

we have a set of files that are used in special cases but are not needed
we use depsys to compile these into the autoload subdirectory.

step 8: algebra (use interpsys)
using interpsys, compile the algebra

step 9: we may have to do deep debugging ==> debugsys
this is purely for my use. sometimes the only way to find a
bug is to run the interpreted lisp code.

Just to add to the pain each of the source files are documents so
we have to extract the required sources at each and every step.

==============================================================
BOOT HISTORY

Boot is necessary to translate the boot files to clisp files.

It is not normally used in a running system. However, boottocl
is a command line tool that allows you to dynamically translate
a boot file to a clisp file. This is a way of making dynamic
changes to the sources (it's lisp, after all). The system used to
be built on top of an existing system. Our style of working was
to keep modifying the system while it was running. Thus the boot
code was changing while we worked. Eventually this led to a system
that could no longer be built from scratch but required a running
system to build. Boot is one example of this problem. The boot
compiler is written in boot. In order to start the process we now
cache the generated lisp files in the original documents. Hacking the
boot compiler would require re-generating and re-caching these files.

The boot code is slowly disappearing from the sources.  Once that
happens the boot subdirectory is no longer necessary and boottocl will
disappear. As you can see from the steps above boot adds complexity
we don't need.

As mentioned previously in these mailing lists Dick Jenks did work
in languages. He created a Meta language to describe the boot syntax
and then a boot language.  This is one of the early attempts to build a
paren-free version of lisp (\begin{religion} similar to python and
many other failed attempts, past and future \end{religion})

Note that not everyone agrees that boot should die as they like
the language. Since there is nothing axiom-specific about the boot
language it would be perfectly possible to fork off a boot language
project and let it live elsewhere.

==============================================================
FUTURE DIRECTION

I'm rewriting and documenting the system internals into straight
common lisp. At the moment I'm working on the interpreter which
will eventually become the 5th volume of the axiom series (hence,
src/interp/bookvol5.pamphlet). This fully documented interpreter
will mirror the existing system. Once it is documented we can
modify it to suit future needs.

My ultimate aim is to make the system a literate lisp program that
only needs to be compiled and run in a single pass on any ansi
standard lisp. The lisp ASDF facility will allow us to remove the
makefiles and the autoconf/autoload machinery. Boot will disappear
as will the many lisp images, intermediate directories, etc. Lisp
will be extended with native notangle/noweave so you can just

The next version of axiom will consist of a set of volumes that are
literate documents, fully explaining the code. The compiler will be
its own volume and can be replaced by a different compiler.  The
interpreter will be its own volume and can also be replaced (possibly
by an aldor standalone version).  The algebra will be organized (in
some fashion) as volumes that fully document that algebra. Axiom will
sit on top of an ansi common lisp literate base (or aldor standalone
base if we go there).

The build process should involve just a drag-and-drop of the volumes
and any literate papers containing algorithms you might want to use.

Tim