Lout initialising run hyphenation and finding wrong file

[Ian Jackson]

(NB crosspost between lout list and debian-devel; trim if appropriate.)

I have a small problem with Lout's behaviour during the initialisation
run (-x) which is called from the makefile's `install' target.

When we build a Debian binary package we do so in a `target' directory
containing /usr, &c, where we put the binaries and other files which
need to go into the actual package.

In order to do this for Lout I set the BINDIR at so forth variables so
that they depend on a variable called `prefix' which is set to `/usr'
in the makefile.  Then I build the lout and c2lout binaries using
plain `make'.  This generates a version of lout with the right,
standard, paths for included and data files.

Then I call `make install prefix=debian/tmp' (debian/tmp is the target
directory).  This correctly copies all the files into
debian/tmp/usr/bin/lout, debian/tmp/usr/lib/lout/* et al.  However, it
then does the special initialising run `lout -x'.

I patch the makefile so that lout -x is given -I, -D, -H etc. in
debian/tmp, eg  lout -x -s -Hdebian/tmp/usr/lib/lout/hyph ...
This makes lout find the files in debian/tmp, rather than the ones in
/usr (which may or may not exist, and certainly aren't guaranteed to
be from the same version of lout or the Debian lout package).

For the .ld files lout correctly sees that (for example)
debian/tmp/usr/lib/include/lout/loutrefs.ld doesn't exist and builds
it from debian/tmp/usr/lib/linclude/lout/loutrefs.

However, for the .lp (packed hyphenation) files lout uses a different
algorithm.  It searches the whole search path for the .lp file and
only if it didn't find it at all does it build it, then generating it
from the .lh file found first in a separate search of the path and
putting it in the first path directory.

In my case this causes lout to find the version of the .lp file in
/usr - if there is one - and conclude that it doesn't need to rebuild
it.  Therefore the package is built _without_ .lp files in
/usr/lib/lout, and when the user installs that version of the package
and then runs lout it doesn't work because it wants to create the .lp
file but doesn't have write permission to /usr/lib/lout/hyph
directory.

In order to solve this problem I have for the moment applied a patch
to the Debian lout sources which make it ignore any existing .lp files
if it's doing the initialising run.  This forces it to (re)create the
.lp files even if there are some somewhere on the search path, whether
in the same directory or another.  I've attached a copy of the patch
below, for information.

There were a number of other solutions I could have chosen:

I could have added a new command-line option to clear all the search
paths.  That would have made lout fail to find the versions of the .lh
files in /usr and had in a way less general impact on lout's file
searching algorithms.  However, the command-option space was already
cluttered and I didn't want to do this without consulting people
here.  On the other hand such an option might be more generally
useful.

I could have had lout check for timestamps or check that the .lp file
existed in the same directory as the .lh file found first on the path
for each .lh file being set up.  However, I felt that this was beyond
my competency with the lout source code (and the patch probably
doesn't add much more of a useful feature than the one I choose to
write in the end).

I could have built a different lout binary with different search paths
to do the initialisation run.  I rejected this because I felt it would
be too prone to error due to other unintended differences between the
two lout binaries.  Also, lout takes a considerable time to build on
my current development machine (a 486DX/33) and I also didn't feel
that the amount of pratting about with makefiles required to build two
versions would be worthwhile.

I could have used Debian's post-installation script facility to build
the .lp files.  However, it is slow and (comparatively) unreliable to
build files at install-time compared to at build-time: instead of me
having to wait once for it, all the users have to wait once each, and
strangenesses of their system can cause them to have deviant versions
of the files.

I could have used chroot.  This was pretty much a non-starter as the
amount of stuff required in the chroot world to get a dynamically
linked ELF binary to run would have been large and complicated to
manage automatically (these files would have to be copied in
automatically and then removed again, by the build scripts, which
would have to determine what they were without necessarily knowing,
for example, the major number of the C library in use).

So: comments on this problem and my solution(s) would be welcome.
Jeffrey Kingston should feel free to include my patch in future Lout
releases, or to add some other facility which would help me, but I'm
not necessarily asking for that.

Thanks,
Ian.

--- lout-3.10.orig/z36.c
+++ lout-3.10/z36.c
@@ -742,7 +742,20 @@
   debug0(DFS, DD, "  calling DefineFile from TrieRead (1)");
   packed_fnum = DefineFile(string(fname), HYPH_PACKED_SUFFIX,
     &fpos(fname), HYPH_PACKED_FILE, HYPH_PATH);
-  packed_fp = OpenFile(packed_fnum, FALSE, FALSE);
+  if( !InitializeAll )
+  { packed_fp = OpenFile(packed_fnum, FALSE, FALSE);
+  }
+  else
+  /* If we're initialising we ignore any packed files.  This is
+   * because we don't check timestamps (like we do for database
+   * files), and so we can accidentally leave an old packed version
+   * lying about rather than generating a new one.  In the case of the
+   * Debian package creation we can find the existing installed
+   * version of the .lp file and so decide we don't need to generate
+   * it !  Anyway, hence this check.  - iwj 26.08.1997
+   */
+  { packed_fp = NULL;
+  }
   if( packed_fp == NULL )
   {
     /* no packed file, so define and open unpacked one instead */