Re: Where to start

[Mike Gran]

On Wed, Feb 27, 2019 at 05:38:23AM +0100, address@hidden wrote:
> I guess, on some level I am asking how to go about familiarizing myself with
> the code. How to approach understanding the logic behind each directory. Is
> there a map that designates the differences between /lib and /libguile? So
> on.
> 
> On 27.02.2019 02:22, address@hidden wrote:
> > Hi there. I think some of you already know me from the Guix mailing
> > lists. I come from a biosciences background, and am transitioning into
> > a career in computer science. I plan on starting my masters this year.
> > A large part of my interest has been in LISPs, and particularly Guile
> > and other implementations of Scheme.
> > 
> > I know that Guile 3 is right around the corner, and i'd like to be
> > able to lend a hand in getting it ready for that. I have experience in
> > both Scheme and C. This would be my first contribution to a C project
> > though. I am curious if there is some resource that explains where I
> > may focus my efforts? I know the bug tracker is available for this,
> > but where else may I focus my attention?
> > 
> > I imagine most of what we need for Guile 3 is in a feature-freeze more
> > or less, so I just want to get my cards in order before I start
> > throwing any patches out.
> > 
> > Best,
> > Brett Gilio
> 

Hello Brett,

Nobody has replied, so I will make an attempt an answering your
question.

Basically, when you look at the directory tree, everything starts with
autotools.

All the various configure.ac, Makefile.am, *.m4, build-aux/*, m4/*
files make up the source code used to generate a configure script.
The autogen.sh script in the main directory creates the configure
script from these files.

Then when you run the configure script, it generates the various
Makefiles, as well as a handful of shell scripts.  Basically anything
in the directory tree with the file extension *.in will be converted
into another file by the configure script.

Most of the configure tests are for the C compatibility library, which
is gnulib, which is what lives in the /lib directory.  The Guile
maintainers don't modify anything in the lib directory, but, rather
just get that code from the gnulib project.  Every year or so, someone
updates the code in the lib directory to the newest code from gnulib.

Gnulib is a project that attempts to correct inconsistency among the
various implementations of the C library, and Guile uses Gnulib to help
keep it running on the various operating systems.

Next up in the build it to make the Guile C library, libguile.so or
libguile.dll, which mostly happens in the libguile directory.  This
library contains all the C code that goes into the guile interpreter.
Many of the low-level Guile procedures are written in C.  There is a
complete virtual machine structure in libguile. And there is a
complete scheme parser in there as well.  Most of the research of
Guile v3 is to make a better virtual machine.

Once libguile.so or libguile.dll is built, it is linked to the 'guile'
or 'guile.exe' executable.  This also happens in the libguile
directory.  This guile executable is a complete guile interpreter,
but, at this point, if you did try to run it, it will be incredibly
slow.  Because at this point, none of the scheme files have been
preprocessed into byte code to make startup speedier.

Now begins the most painful part of the build to watch.  This guile
interpreter is used to compile the scheme files that make up the
higher-level layers of Guile into bytecode.  The scheme files are
*.scm; the compiled bytecode files are *.go.  Each time the guile
interpreter runs at this point, first it must load up all of the *.scm
files that make up the higher-level layers of guile without the
benefit of having *.go files, so it is very slow.  But, step by step,
the scheme files that make up the higher-level layers are compiled, so
each attempt to compile a scheme file at this part of the build gets
faster and faster.

Most of the important scheme files that make up the higher levels of
the Guile interpreter are in the modules/ directory, with
modules/system/base and modules/ice-9 being of particular importance.

The absolute slowest part of the build is the compilation of psyntax.
Psyntax holds the syntax and grammar of some parts of the scheme
compiler.

To help try to fix the slowness of the build, there may be a bootstrap/
directory, which contains pre-compiled versions of some core scheme files.
But this trick only works if you haven't modified the compiler in a way
that invalidates these pre-compiled scheme files.

In the test-suite directory are a bunch of tests.  Some are in C and
require compilation.  Most are scheme scripts in the test-suite/tests
directory.

In the meta directory are scripts that help you run guile from the
build tree without a proper install.

So looks like I mentioned lib/, libguile/, meta/, test-suite/, and
modules, so that's the bulk of things that go into the build.

Regards,

Mike Gran