Re: [Axiom-developer] Heidegger, literate programming, and communication

[Gregg Reynolds]

Howdy Tim,

On Thu, May 22, 2014 at 1:16 AM, u1204 <address@hidden> wrote:

Gregg and Gary,

I understand where you are coming from. Indeed, Maturana [0] is on your
side of the debate. Since even the philosophers can't agree, I doubt we
will find a common ground.

Ah, but philosophers never agree.  Disagreement is part of their job description.  Why should programmers be any different?

 

Unfortunately, I've decided to take on the task of documenting the
Clojure internals because, yaknow, *I* don't feel I understand something
until I know what the hardware does; consider this a flaw in my
personality :-)

I suffer from a similar malady, which compels me to continually rewrite other peoples code, since, gee whiz, "foo" is not quite the perfect name for that darn variable, "bar" would be just slightly better, and on and on.  You can see why I prefer code to commentary.
...
>>
>> 1. I think that Gregg Reynolds and I agree on a lot, but I would add to
>> his remarks that there is almost always a human audience for source code,
>> as well as the compiler/interpreter.  Sometimes, the audience is just the
>> originally programmer, perhaps at a later date.  (If I missed something,
>> Gregg, sorry, but I don't think you disagree, anyway.)
>>
>
>I agree; whoever writes the code automatically forms an "audience" of one.
>I guess I would say "reader/responder".

Hmmm. Common Lisp is about 25 years old. Assume Clojure lives that long.
What are the odds that the original authors will be maintaining the
code? Will the code still be "an audience of one"? Are you sure that's
a worthwhile goal?

I think you may have misunderstood me (dunno about Gary): my point is that even one-off code that gets discarded immediately has a human reader, namely the author.  A statement of (minimal) fact, not a goal.

...

   Sophisticated Lisp programmers use macros A LOT.

That's because they are language designers, and they know it.

 

   expanding into macros expanding into macros. Quick, what does the
   spadcall macro do?

HCF?

>Exactly.  Conclusion: it's hard, maybe impossible, to generalize about what
>all code should look like.  Maybe it's essentially pluralistic.

Yes, it is hard to generalize about what all code should look like. But
it is not hard to generalize that reading natural language explanations
is faster, more accurate, and a lot easier than reverse-engineering
code.

Whoa Nelly!  I don't agree with that at all, either in principle or by experience.  Well, ok, you've rigged the game.  Easier than "reverse-engineering code" - what does that mean?  I guess you mean reading well-written natural language explanations is faster etc. than reading badly written code - but so what?  It's not a meaningful comparison.  Would you take a comparison between a sample of well-written code and a sample badly written LP as evidence against LP?  I would not.  To me the question is whether well-written natural language explanation adds anything of substance to well-written code.

 

It is MUCH easier to understand Greg Humphrey's rendering code
than it is to understand the Clojure internals.

Ok, but I don't see how exhibiting a piece of transparent code next to a piece of opaque code demonstrates anything.

 

Consider a randomly chosen paragraph from Physically Based Rendering
(p356):

  To do the permutation, this function loops over the samples, randomly
  permuting the sample points in one dimension at a time. Note that this
  is a different permutation than the earlier Shuffle() routine: that
  routine does one permutation, keeping all nDim sample points in each
  sample together, while here nDim separate permutations of a single
  dimension at a time are done. (Figure 7.21)

   for (uint32_t i = 0; i < nDim; ++1) {
     for (uint32_t j = 0; j < nSamples; ++j) {
       uint32_t other = j + (rng.RandomUInt() % (nSamples - j));
       swap(samples[nDim + j + i], samples[nDim * other + i]);
     }
   }

  Footnote: While it's not necessary to permute the first dimension of
  the LHS pattern, the implementation here does so anyway since making
  the elements of the first dimension be randomly ordered means that LHS
  patterns can be used in conjunction with sampling patterns from other
  sources without danger of correlation between their sample points.

So we learned what the code does. We also learned not to "optimize the
code" by replacing it with Shuffle(). Further, we learned that we
shouldn't "optimize the code" by removing the apparently useless
shuffle of the first dimension. And, as a bonus, we get a figure.
NONE OF THIS INFORMATION IS IN THE CODE ITSELF.

For "what the code does":

/* inline random, total, in-place matrix permutation - contrast Shuffle() */

...code...

The first line of commentary is totally redundant, so it is a waste of time to read both it and the code.  As for what not to "optimize": if none of the conclusions you draw are in the code, they're not in the commentary, either.  Replacing the code with Shuffle would change the meaning from total to partial permutation, so it would not be an optimization.  Ditto for replacing it with Shuffle.  But the explanation of the difference between this code and Shuffle is unhelpful to me - what does "[the Shuffle] routine does one permutation, keeping all nDim sample points in each sample together" mean?  I'd have to read that code to find out, so the commentary has just wasted my time.   As for the permutation of the first dimension, here the commentary is bad again - first it says it is not necessary, then it explains that it is necessary if you want to use it for a specific purpose.  As for the figure, I assume the book has some kind of graphic.  I don't think I need a graphic to understand the code - it's a simple permutation.

So for me at least, this example counts against rather than for the point that (I assume) you want to make.  The point I will make is not that documentation is inherently evil - far from it - but merely that explanatory prose is not a *necesssary* condition of readable code.  And it often makes things worse, as in this case.

 

In addition, this code lives in an organizing structure. It is
in Chapter 7: Sampling and Reconstruction
     Section 7.3: Stratified Samples


Heck, it is only 4 lines of C++. Why bother? *I* can read C++.  I can
even reverse engineer it (probably by inventing the diagram in Figure
2.7 on a napkin). Maybe it lives in the src/SamRecon/StratSam, which is
all the organization necessary. :-)  But I can't reverse engineer the
important information in either paragraph of text.

For comparison, refer back to the multi-page Java code I posted
from the Clojure core. As a maintenance programmer, which would you
rather maintain?

Really?  What's the point of comparing 4 lines of C++ and multiple pages of Java?  I can tell you I would rather maintain the four lines of C++ without the largely useless commentary.  I would also rather maintain a (well-written) plain Pascal version of "Printing the primes" than the LP version Knuth uses in his paper on LP (a web search will turn up a preprint copy).

 

... snip ...

Languages come into fashion all the time. Almost all of them disappear.
Axiom and Clojure are worthwhile efforts. We really need a mindset that
extends into the future, communicating with the maintainers.

Dunno anything about Axiom, I'm afraid.  As for the future, I think it is best served by doing our best to write clear *code* now.  You mentioned Axiom has a 30 year timeline.  I won't be surprised if humans do relatively little hands-on coding (or code reading) by then.  Between AI and automated reasoning (for which the commentary part of LP is irrelevant) I expect the number of programmer jobs to shrink drastically by then.

In any case, I wish you luck with your Clojure internals project.  If XML doesn't give you the hives check out https://github.com/mobileink/xsl.clj-ext - I think it could be used to good effect to document the APIs, if not the internals, by example.

-Gregg