Re: [Axiom-developer] Terms of Surrender (was: BAD tim)

[root]

> First about Boot. My (admittedly poor) understanding of Boot is that it is an
> intermedate language between Lisp and Spad (and there are other intermediate
> languages Shoe, Meta, etc as well). I agree that to build a truely high level
> language like Spad/Axiom/Aldor, intermediate languages are necessities and not
> evils. It's much like what a mathematician does all the time (and most 
> original
> developers are mathematicians, in addition to being computer scientists): if 
> the
.....[snip].....

sorry, but you are reasoning by wrong analogy.

boot is not an intermediate step between lisp and spad. boot is a 
syntactic sugar coating on lisp that eliminates parentheses in favor
of "piling" (indenting) code. it adds no new functionality and does
not help you reason about programming any more than, say, python or perl.
in fact it limits the things you can write so you can't use the full
power of lisp.

boot does not function as a lemma-like level of reasoning.

> Second, I think once the bootstrap step is done, it is ok to write a compiler
> for the language in the language itself. So writing a compiler for C in C, a
> precompiler for C++ in C, one for Boot in Boot, or for Axiom in Axiom is
> perfectly reasonable. The purist may want to boil everything down to
> hand-crafted assembly (dare I say machine?) language, 

.....[snip].....

swing and a miss, as they say in baseball. 

the interpreter needs to be:
   simple to build, 
   simple to maintain, 
   simple to extend, 
   simple to port.

writing boot in boot forces the system build to be more complex. 

it forces the maintenance to be harder since you need to understand 
whether the problem is in the boot compiler or the interpreter. 

it forces the user to understand the interaction of lisp vs boot 
when extending the system since certain facilities have to be in 
lisp because boot is not yet available at that point in the build 
process (e.g. setvars)

it makes the system harder to port because we have to rewrite the
bootstrap lisp code from old common lisp to ansi common lisp to port it.
this makes the boot-level code out of date.

writing boot in boot is clever but it makes building, maintaining, 
extending, and porting axiom harder than it has to be.

yes, this can be fixed but fixing it means that you have to
  (1) rewrite the boot compiler changes in boot
  (2) recompile the boot compiler to clisp
  (3) clip out the changes from the clisp file
  (4) insert the changes into the bootstrap sections of the boot compiler
  (5) rebuild the system

writing boot in boot is clever.
we don't need clever code.

boot is NOT a higher level language than lisp. spad is.
boot is NOT spad and is unrelated to spad.
boot is ONLY seen by interpreter/compiler developers.
boot is used by 1 developer (me) until the documentation completes.



> > the other "oh by the way" is that while you can seem to
> > read boot code without knowing lisp (a sort-of pseudo
> > code syntax) you can't WRITE boot code without knowing
> > lisp.
> 
> Bill page responded:
> 
> > Why is that? Surely it is possible to write Spad code without
> > knowing Lisp? 

because boot IS a limited version of lisp. for instance, in the
history mechanism in bookvol5.pamphlet you'll see that boot uses RPLAC
functions to create a circular list. to write this you need to
understand that (a) you're using list structures, (b) how they are
represented (as cons cells) and (c) how rplaca/rplacd modifies
them. all of that requires an understanding of lisp, not boot.

> > 
> > Now about this language war ...
> > [snipped] 
> > So, *if* I was thinking of such a compromise then there are
> > some "terms of surrender" that might make this easier:
> > 
> > 1) Before any more "lispifying" goes on, *first* move Axiom
> >    over to ANSI lisp. This will make it possible to write lisp
> >    that conforms to modern practices and as high level as
> >    possible. It will also make Axiom less dependent on a
> >    particular implementation of Lisp and more easily portable.

so you're suggesting that we modify the boot compiler to generate ansi
lisp code? yet during the "lispifying" rewrite i AM rewriting to ansi
lisp.

> > 
> > 2) No more just cutting and pasting the generated lisp output
> >    of Boot into the literate document! This seems pointless
> >    and a net loss in intelligibility even if it is accompanied
> >    by a verbal explanation of the code. The original Boot code
> >    is, as Tim admitted above, significantly more readable as
> >    "pseudo-code" then what the compiler outputs.

the cut and paste has to happen. it's the only way to make sure the
code does what it originally did. successive passes of the code have
been complete rewrites. you'll see this at the next code release.
there are a LOT of subtle issues that require careful thought and
having working lisp code, even clipped code, is a necessary guide.

believe me, i find the boot compiler output code even more painful
than anyone else. but it is important to move the code from a
correct state to a correct state during the rewrite and i'm not
man enough to do it all in one go.

(btw, as a side-effect of the rewrite i've already found and
corrected two bugs).

if you tried to maintain the system as it is now you'll find that
the boot compiler output is all over the source tree. try this:

  cd axiom/src
  fgrep -r DEFUN *

the boot compiler generates an uppercase DEFUN. look at how
pervasive the compiled lisp code has to be to actually get the
system to bootstrap. in every file where the DEFUN occurs
the BOOT code is just PSUEDO-CODE.

in particular, 

  cd axiom/src/boot
  fgrep DEFUN *

and you'll see how many of the boot compiler files aren't REALLY
written in boot.

believe me, it took a long time with a lot of effort to get the
system to build without a running axiom system.

 and it ain't pretty. but it should be.

> > 
> > 3) Where possible and practical, replace Boot code with Spad
> >    code as in Dick Jenks original vision instead of Lisp. Surely
> >    you are not going to argue that Spad is a "dead language"
> >    written by fewer than 10 surviving programmers? And even if
> >    that happens to be true ;), that is one thing that we are
> >    trying hard to change, right?

wrong level. boot is NOT spad. the languages are NOT related except
by authors. they have different design goals. 

> 
> These terms should not be negotiable! Actually, I would put these as 
> requests to whoever does the work) I would add at least 

ummm, i'm doing the work :-)

and i REALLY don't want to rewrite the boot compiler to make it do
things like ansi common lisp generation. especially if, after having
achieved the 3 *terms* i'd would then have agreement that i could
elide boot from the system. what would be the point?

> 
> 4) The error messages outputted by the compiler should NOT refer to any Lisp.
> Nothing like the current $m, $x, $f stuff. All errors should refer as close to
> the offending source code as possible. (I believe such output was due to
> "laziness" or viewing more intelligible messages as an insult to the 
> developers.
> But that might be why Axiom failed to capture market shares in the 1990's.)

the output uses the variables like $m because that's all the developers
need to know to debug. $m is the target mode information that the
compiler is trying to achieve. 

i agree that the compiler needs to be rewritten to do things like
generate better error messages. 
  to do THAT level of rewrite requires a deep understanding of the compiler. 
    to achieve THAT level of understanding we need to deeply document how 
    the compiler works. 
      to do THAT level of documentation we need to review each function 
      in detail. 
        to do THAT detailed level of documentation is my current goal. 
        as a side-effect of achieving that level of understanding the
        code will be in lisp (which is a language i deeply understand.)

> 
> 5) The )trace function should likewise return high level state of the algebra
> variables, not lisp code or representation of them.

admirable goal. BIG changes to the interpreter and compiler data
structures (since the code is COMPILED into machine language) to
maintain state information that trace can use.

and it needs to be done without impacting efficiency, accumulating
large amounts of possibly unneeded state, etc.

this is like asking C to give you only algorithmicly related error
messages. 


> 6) The interpreted language should adhere to the compiler language to 
> facilitate
> prototyping (in the interpreter) and production code (using the compiler) and
> any deviations (due to the smartness of the interpreter) should be logged to
> help the transfer (maybe it is even possible to translate a working 
> interpreted
> set of routines automatically into compilable code?).

we need to "reify" (name) the interpreter type-translation, guessing
and manipulation mechanisms. we need to separate them from the
interpreter itself so we can understand, add or modify them. 

( to do THAT level of rewrite requires a deep understanding of the interpreter
  ( to achieve THAT level of understanding we need to deeply document how 
    the interpreter works. 
     ( to do THAT level of documentation we need to review each function 
       in detail. 
       ( to do THAT detailed level of documentation is my current goal. 
         as a side-effect of achieving that level of understanding the
         code will be in lisp (which is a language i deeply understand.)))))


read the book when it comes out.
the story turns out ok in the end.

BAD Tim