Update on Gash and Gash-Utils

[Timothy Sample]

Hi all,

This is an update on porting Gash and friends to Mes.  Here’s my latest
branch: https://git.ngyro.com/mes/log/?h=wip-gash-utils

It adds another 62 commits on top of ‘wip-gash’.  Git says:

> 60 files changed, 2761 insertions(+), 183 deletions(-)

With these changes, I am able to use Mes, Gash, and Gash-Utils to
configure, build, and install packages!  The most glaring shortcoming is
that I had to punt on Awk by including Gawk in the test environment.
Good thing we have a long history of building old versions of Gawk!  ;)


New stuff
---------

The main improvement is probably the new system interfaces.  We now have
‘stat’, ‘lstat’, ‘rename-file’, ‘mkdir’, ‘rmdir’, ‘umask’, ‘utime’
(almost – see below), ‘sleep’, ‘link’, and ‘symlink’.

I added support for regular expressions by bringing in Dorai Sitaram’s
pregexp (Portable regular expressions for Scheme) code.  I had to make a
minor adjustment to it, and wrap it in a Guile-like interface.  That
interface also has a little hack to deal with “basic” regular expression
syntax.  Grep and Sed seem to be happy with it.

I also added true bytevector support.  This means is that Scheme can
manipulate ‘TBYTES’ objects via ‘make-bytevector’, ‘bytevector-u8-ref’,
and ‘bytevector-u8-set!’.  I added bytevector I/O procedures, too, but
they are written in terms of reading and writing bytes (that is, they
don’t take advantage of the fact that we have contiguous buffers to do
fast I/O).  One fallout of this is that I deleted MesCC’s pretend
bytevectors.  Rather than rewrite them on top of true bytevectors, I
just had MesCC make lists directly.  The reason for this is that it
assumes the objects made by the pretend bytevector interface are lists
later on!

We have more code from Guile, too.  SRFI 37 gives us ‘args-fold’, and I
was able to mostly adapt ‘file-system-fold’ to work on the directory
support added in the last round for Gash.

A few smaller items include ‘sort’ (via a SRFI 132 stub), ‘and-let*’,
setters for immutable records, and ‘set-fields’.  There are more list
and string procedures and bug fixes, but you can read them in the commit
log.

Lastly, I made modules the default and disabled the “psyntax” test like
we talked about.


Problems
--------

When I wrote ‘utime’, I wrote it in terms of the ‘utimes’ system call.
Now I’ve learned that this is an older system call that is not supported
of RISC-V.  I will have to rewrite it in terms of ‘utimensat’ (which is
not hard, but I don’t have time for it ATM).

There’s a weird ‘xxx! multi-squash!’ commit.  There was a bug in the
changes I made to ‘primitive-load’.  This commit needs to be split up
and squashed in a few places.  I can do it, but we should coordinate on
the history rewrite.

Awk is going to be a bear to port.  In Awk, I rebind ‘let’ and ‘let*’ to
have SRFI 71 semantics (nicer syntax for ‘let-values’, essentially).
However, Mes has very loose ideas about macro binding.  Rebinding ‘let’
in Mes seems to have global effects.  This introduces a circular
dependency.  What I want: my ‘let’ → ‘let-values’ → normal ‘let’.
However, on Mes my ‘let’ and normal ‘let’ are unified.  I’m sure there’s
a way to hack around this, but – to be frank – I’m tired of “hacking
around”.  I’m trying to decide if I could find it in me to roll up my
sleeves and fix the root problem here....  (Guix code will be very hard
to run without solid macro support.)


Questions
---------

Besides the bytevector stuff, I wrote some weird code to expose bindings
in the root module.  See ‘define-in-root’ in “boot-5.mes”.  I could
follow this pattern to expose more things in the root module while
having the benefit of the module system to define them.  For example,
instead of defining ‘string-every’ in “srfi-13.mes” and manually
unpacking the optional arguments, I could define it using ‘define*’ in
“srfi-13.scm” (using whatever modules I want) and then expose it in the
root module.  Thoughts on this approach?  Is the syntax too much?
Something like this would really help with Guile compatibility in the
long term (and simplify the Gash and Gash-Utils Mes shims in the short
term).


Conclusion
----------

I’m assuming the next steps (to get from 0.25 to 0.26) will be to
integrate this work into Guix.  My idea for that is to get to a point
where, after Mes is built, we do the rest of the bootstrap with Gash and
Gash-Utils on top of Mes (using ‘%bootstrap-guile’ only for
orchestrating the build scripts).  With that working, we could
coordinate releasing versions of Mes, Gash, and Gash-Utils that support
that outcome.  Of course, I’m happy to defer to the maintainer if there
are other plans.  ;)

Lastly, FYI, the relevant branches for Gash and Gash-Utils
‘wip-modular-mes’ and ‘wip-mes’ respectively (on Savannah).


-- Tim