[Axiom-developer] Re: Build machinery -- support for out of source build

[Gabriel Dos Reis]

On Mon, 2 Jul 2007, address@hidden wrote:

| Clearly you don't see that a compromise is worth attempting.
| That's fine. It was just a suggestion.

If you believe I'm not at the compromise table, then we should probably
stop this discussion.

[...]

| > The argument that 'by design the "src" tree should never be
| > touched by Make so you should be able to mount the source read-only
| > and type Make' is inherently bogus. "src", "obj" and friends
| > are created *within* the Axiom source tree.
| 
| Ok. I guess I wasn't clear enough. Let me try again.

Oh no, you were clear.  But you keep saying "I believe", when
you are invited to try for *concrete* an experiment.  Please could you rry the
experiment I suggested before you continue?

[...]

| simple bugs, I'm sure. However, the design calls for the construction of 
| 
|   /home/gaby/int
|   /home/gaby/obj/linux
|   /home/gaby/mnt/linux
| 
| which, as I understand it, is an out-of-source build.

What is out-of-source build is this:

   (1) check out Axiom into ax-src
   (2) mkdir ax-build
   (3) cd ax-build && ../ax-src/configure && make

Report back.

   Try similar experiments with GCC, GDB, Autoconf, Automake, binutils,
fileutils, etc.  Report back.

What are the logical reasons why Axiom should refuse to support that?

[...]

| * Minimize work
| 
| > In build-improvements, the dependency is slightly more
| > complicated because I have added a full bootstrap in three stages.
| 
| That's fine but uninteresting. The goal of making autoconf work in
| silver has nothing to do with your boot improvements.

You are wrong.  A dependency tracking needs to be part of the system,
not just at the Make level, but also the system level.  Otherwise, you get
into obscure contorsions as exemplified by the build of depsys. 

[Please, don't say I don't understand why the build of depsys is the way you
wrote.  However, that is only because you chosed a poor tool to do a poor
implementation]. 


[...]

| > The argument that "int/" speeds up the build process is almost
| > fallacious.
| 
| I listed 5 ways that "int" speeds up the build process. Why don't those
| ways speed up the build? The "int" directory cache doesn't speed up
| the build the first time but it doesn't slow it down either. However,
| the second build is much, much faster if you retain the cache.

Have you tried setting up a dependency of the files at Make level, and try
make -j N, where N is any positive integer of your choice? N=4,8,16
usually gives me bindly fast build on build-improvements.


[...]

| You could argue that the files in "int" and the files in "obj" could
| be merged into a single directory with the same speedup result. That
| is true in terms of speed. However you lose one major ability when
| you merge int and obj. Suppose that we want to build Axiom on several
| different platforms, e.g. fedora, debian, ubuntu, solaris, and other
| systems that might exist in a compile farm. In the current split the
|   
|    int directory contains system-independent, machine-generated files
|    obj directory contains system-dependent, machine-generated files
| 
| so you can
|   
|   1) do a complete build on one system
|     for each different system do:
|       2) copy the whole axiom tree to the next system
|       3) rm -rf obj mnt
|       4) set the AXIOM variable
|       5) make

I do not think I lose that ability; see below.

A note:
Have tried that for a fact  *on today systems* to know that the above scheme
actually speeds up the build compared to what we have in build-improvements?
I have not, but here are a few points:

   (1) I tried separate concurrent builds on many machines since 2005.
   (2) Waiting for one system to complete is a complete waste of valuable
       time, because by the time one completes, it is likely that all
       have already completed -- and I've seen that many times.  Therefore the
       time and effort to copy to other systems are pure waste.  If you have
       30 years before you to build the system, that is fine.  Many 
       potential contributors haven't.

   (3) How often do you need to know that today?  Design for common usage.


Here is why I don't think you lose the ability you mention above: once you
have a complete build, you can just issue make clean-objects [or any make
target of your choice]; that will automaticaly clean all system-dependent
files.   

But above all, the simplest, and probably fastest way on today machines,
is to simply support simple out-of-source build as I outlined above,
parallel make (make -j N)


| I'm not sure how you might do this if int and obj are merged.

Have make target for them.  Simple.

Today on build-improvements, I can step into a subdirectory and issue
make, make clean, etc. and have the specific task I want done by Make.

| You clearly have a different idea of how autoconf should structure
| system builds in silver.

It is wrong to think that I believe Autoconf should structure
system builds in silver.  It does not structure the build.  Its sole
purpose is to collect system-dependent information and generate a few
(source) system dependent code from a single template.  That is all.
The rest of the build is conventional structured Makefiles.  There is
nothing fancy in Axiom that needs any of the unconventional obscure setup we
have in Silver.  

If you want a maintainable system and to facilitate learning of the system by
potential contributors, you don't want to be creative where a simple standard
construct does the job, and better. 

| That's fine. I'm simply pointing out that
| the current build system has features which most people do not 
| understand.

Instead of saying tha people don't understand them, it might be accurate to
say that people understand the ideas but disagree with the implementation.

-- Gaby