Re: [Axiom-developer] RE: cross-compiling Axiom

[Camm Maguire]

Greetings!

"Bill Page" <address@hidden> writes:

> On September 3, 2006 8:02 PM Tim Daly wrote:
> > 
> > Axiom was ported to the Zaurus handheld using cross-compilation.
> > Camm does it in the Debian setup.
> 
> No, I don't think that is true. At least I know it isn't true
> for the version of Axiom that I run on my Zaurus. Axiom is built
> on Debian like all other packages that I know of on Debian - on
> a machine that is hardware-compatible with the target.
> 
> http://db.debian.org/machines.cgi
> 
> I think the binary that I use on my Zaurus was compiled on Debian
> running on such an Arm host. (Camm can confirm.)
> 

This is correct.

> http://www.debian.org/ports/arm
> 
> These machines have the same type of processor as the Zaurus
> (although of course these are probably not handheld devices :)
> having faster processors, more memory and faster mass storage.
> 
> In order to run Axiom, I must also configure my Zaurus to run
> a similar Debian distribution. On my Zaurus the arm Debian
> distribution runs in a chroot.
> 
> > 
> > There are two methods generally used, CROSS-COMPILATION and
> > CROSS-MOUNTING. 
> >
> 
> I have never heard of "cross-mounting" used in this context.
>  
> > 
> > CROSS-COMPILATION involves two systems, B -- the build system
> > and T -- the target system. You compile on B but tell GCC that
> > the target is T. 
> > 
> > GCC is designed in layers. The source code eventually gets
> > compiled to RTL (Register Transfer Language) which is a target
> > independent lisp-like assembler language. Since this is system-
> > independent you can use the RTL anywhere.
> > 
> > The final stage of GCC compiles RTL to the target platform. GCC
> > currently targets about 20 platforms.  This is useful for targets
> > like the Zaurus (an ARM (i386-ish) processor) that do not have
> > the horsepower to be a compile platform. It is also useful for
> > embedded systems.
> >
> 
> In fact I have built Axiom from source on my Zaurus, but I do not
> recommend it. Since there is not enough main memory you must
> configure a swap disk and swapping to SD ram is *slow*. It took
> about two days to complete.
> 
> Perhaps it is possible to cross-compile programs written in C to
> arm using gcc. For example, here is an ambitious project of that
> type:
> 
> http://scratchbox.org/wiki/Crocodile
> 
> But I am quite sure that this is *not* how Camm compiles Axiom for
> Arm and the other Debian architectures.
> 

While I am certainly not the expert here, I do know that Debian in
particular has given considerable thought to portability.
Traditionally, it has supported synchronous releases on 11 different
architectures -- a truly massive feat, especially when considering the
size of the archive.

As time has progressed, the spread between the fastest and slowest
machines has widened, and this has sparked some dicussion on the
development mailing lists regarding a possible  two-tier
classification system for supported arches into 'main' and 'scc'
("second-class citizen").  Some such moves are inevitable I suppose at
some point, but I just want to emphasize how many bugs have been shaken
out of gcl supporting all these different machines, how much bloat had
to be remmoved to make it feasible, and how much remarkable support
has been offered from the respective supporters of the particular
architectures.  The m68k crowd (aka former amiga users) is
particularly amazing.

In this discussion, some consideration was given to constructing
cross-compiling auto-builders.  While I did not follow it to the end,
the consensus appeared to be that the technology was not yet
sufficiently fool-proof.  You can dig this up on the debian-devel
archives if interested.

> > 
> > CROSS-MOUNTING is easier if the target system has the tools and
> > horsepower to host GCC. For instance, to build on a Sun system
> > it is easier to use NFS (Network File System). Here we talk about
> > S -- the source system and T -- the target system.
> > 
> > On S, which contains the source tree, you 'export' the filesystem
> > using NFS. This allows anyone to mount the filesystem remotely. On
> > T you NFS-mount the sources. They now appear to be on the target
> > system and you can just type 'make'
> 
> Tim, that sounds like nonsense. Anyone can mount source directories
> located on another machine running another operating system. You can
> use NFS, or Samba or whatever. This has nothing to do with cross-
> compilation.
> 
> > 
> > Axiom can use this system. First you build on S so that you have a 
> > full build tree (SRC, INT, OBJ, MNT). Notice that these are defined
> > as
> > 
> >   SRC -- Human created, read-only
> >   INT -- Machine-generated, Machine-independent
> >   OBJ -- Machine-generated, Machine-dependent
> >   MNT -- Final distribution tree
> > 
> > 
> > SRC contains the source tree. NFS export on B it so it can be 
> > mounted on T. INT contains a 'cache' of extracted code that is 
> > system-independent so it also can be NFS exported on B and
> > NFS-mounted on T. This saves having to redo a lot of work.
> > 
> > On B you export SRC & INT.
> > On T you NFS-mount SRC & INT.
> > On T you set the AXIOM variable to the local machine 
> >   (e.g. export AXIOM=`pwd`/mnt/sun)
> > On T you type 'make' and it builds a Sun version.
> > 
> > In parallel, other systems can also do NFS-mount and make.
> > The only change is the AXIOM variable (e.g. `pwd`/mnt/macox)
> >
> 
> Maybe this strategy was interesting once-upon-a-time but with
> the machines that we have available today I doubt that there is
> much advantage to this. It seems to me that all you might save
> is the BOOT-to-Lisp and SPAD-to-Lisp compile times. The only code
> in Axiom that is (approximately) machine independent is the lisp
> source code and the generated lisp code. To get anywhere else
> with that you still need a lisp interpreter/compiler on the target
> environment. And it seems to me that that is the hard part.
>  
> > 
> > The advantage of CROSS-COMPILATION is that you can build for
> > systems that don't support a build environment. The disadvantage
> > is that you need a cross-compiled environment (e.g. libraries
> > need to be cross compiled, .h files need to reflect the target,
> > etc.). Since Axiom needs the X11 libraries for a full build you
> > need to have X11 libraries that are cross-compiled for the target
> > (e.g. Zaurus)
> 
> The version of Zaurus that I have (which still runs the Sharp
> supplied embedded Linux) does not support X11 natively. That is
> one reason why you need the Debian arm port. There may be a
> version of OpenZaurus Linux that does have X11.
> 
> But I still do not understand how you can talk about cross-
> compilation and GCL. When GCL compiles some lisp source code
> it produces intermediate C code which, yes could in principle
> be compiled for another architecture, but then it immediately
> attempts to link the generated object code into the running
> GCL image. If you are attempting cross-compilation this is
> not possible because the running GCL image is not compatible
> with the object code produced by gcc.
> 
> What am I missing?
> 

Nothing AFAICT in the existing build framework.  I just want to
mention that GCL unlike some other lisp systems supports both an
interpreter and a compiler.  This is primarily to aid in developing
and debugging the compiler, but also provides some means for platform
independent work.  The whole Axiom system should load and run
interpreted if desired, and if one has a massive amount of time on
one's hands :-).  The same holds for the GCL lisp files -- check out
the saved_pre_gcl image.

One item of medium term interest in GCL development is a just-in-time
byte-code compiler/interpreter for faster platform-independent
(i.e. non-native code) execution.  Few have expressed an interest in
this thus far.

> > 
> > The advantage of CROSS-MOUNTING is that you are building on
> > the final target and can use all of the libraries, includes,
> > and tools. The disadvantage is that you can't do this if the
> > target does not have the horsepower.
> 
> As far as I can see, this is irrelevant to the issue of cross-
> compilation.
> 
> > 
> > Which method you use is determined by the resources of the
> > target.
> > 
> 
> If you build the binaries on the target this is not cross-
> compilation.
> 
> With regards to cross-compilation and Maxima see for example:
> 
> http://www.math.utexas.edu/pipermail/maxima/2003/003434.html
> 
> Camm writes:
> 
> > I believe you will run into some problems when cross-compiling.
> > You can compile all the objects just fine by pointing gcl at
> > your gcc cross compiler.  But part of gcl's build, and maxima's,
> > entails loading and executing these objects (i.e. initializing
> > the lisp core), so you would at least need a mips emulator, which
> > I doubt exists. Your best bet I'd think would be to setup your
> > pda to mount a larger filesystem via nfs from another box, with
> > a true mips development environment installed.  Compile there,
> > and your pda should be able to run the produced executable. 
> 
> http://coding.derkeiler.com/Archive/Lisp/comp.lang.lisp/2005-04/msg01510.htm
> l
> http://coding.derkeiler.com/Archive/Lisp/comp.lang.lisp/2005-04/msg01463.htm
> l
> 

I think one can reduce the problem down to cross-compiling C code
produced by gcl/axiom running interpreted.

Take care,

> 
> Regards,
> Bill Page.
> 
> 
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah