Re: chicken-doc instructions recommend extracting tar file as root

[Lassi Kortela]

> For a proper fix, could chicken-doc be modified to download the tar 
> file, sanity-check its contents, and unpack it safely into the user's 
> home directory instead?
>
> Alternatively, if the documentation is shipped in some kind of file 
> format with an index for fast lookup, it doesn't need to be extracted 
> into multiple files at all. There are reasonably simple databases like 
> CDB and Berkeley DB for jobs like this.

CDB is a very simple format that could be ported to Scheme. There's a 
Common Lisp library for it here: https://github.com/xach/zcdb

Alternatively, I tried

tar --to-stdout -xvf chicken-doc-repo-5.tgz >chicken-doc-repo-5.scm 2>&1

and it produced a 12 MiB file. (The -v and 2>&1 cause tar to output the 
filenames of the extracted files into the dump in addition to the 
contents of those files.)

The following script slurrrp.scm:

(define (read-all)
  (let loop ()
    (let ((form (read)))
      (unless (eof-object? form)
        (loop)))))

(with-input-from-file "chicken-doc-repo-5.scm" read-all)

clocked using `time csi -script slurrrp.scm` rips through the file in 2 
seconds on average on my computer. This could be acceptable performance.

What you're storing in the tar file is just 2000 small S-expression 
files. You could vastly simplify the system by concatenating those files 
into one. You can still gzip it too if you want to, though does 12 MiB 
(uncompressed) vs 2 MiB (gzipped) really matter? "What's an order of 
magnitude between friends?" as the saying goes.

The chicken-doc wiki page has another section:

> Cleaning up old repository crust
> Occasionally, or when the repository format changes significantly, you should 
> wipe out your repository before extracting a new one, to get rid of dead wood. 
> Simply delete the directory shown by the following command:
>
> $ csi -R chicken-doc -p "(locate-repository)"
> /usr/local/share/chicken/chicken-doc
> If you have chicken-doc-admin installed, just do instead:
>
> $ chicken-doc-admin -D

That cleanup is not needed if the whole thing is just one S-expression 
file instead of 2000 files in a tarball.

Yet another solution is to ship the tarball but store it as-is and have 
chicken-doc extract it in memory instead of having the user extract it 
manually. This is not hard either: uncompressed tar is a simple format 
and you doubtless have a Scheme library for it in the egg collection.

But it seems shipping one big S-expression file would be simplest.