[bug#33215] [PATCH 06/11] build-system: Add 'clojure-build-system'.

[Alex Vong]

From c7a763cfa2543119cc707b434efc8680f73f6367 Mon Sep 17 00:00:00 2001
From: Alex Vong <address@hidden>
Date: Thu, 25 Oct 2018 05:44:01 +0800
Subject: [PATCH 06/11] build-system: Add 'clojure-build-system'.

* guix/build-system/clojure.scm, guix/build/clojure-build-system.scm: New
files.
* guix/build/clojure-utils.scm (@*, @@*): New macros.
(%source-dirs, %test-dirs, %compile-dir, %main-class, %omit-source?,
%aot-include, %aot-exclude, %tests?, %test-include, %test-exclude,
%clojure-regex): New variables.
(package-name->jar-names, canonicalize-relative-path, find-files*,
file-sans-extension, relative-path->clojure-lib-string, find-clojure-libs,
compiled-from?, include-list\exclude-list, eval-with-clojure, create-jar):
New procedures.
* Makefile.am (MODULES): Add them.
* doc/guix.texi (Build Systems): Document 'clojure-build-system'.
---
 Makefile.am                         |   2 +
 doc/guix.texi                       |  58 ++++++++
 guix/build-system/clojure.scm       | 195 ++++++++++++++++++++++++++
 guix/build/clojure-build-system.scm | 110 +++++++++++++++
 guix/build/clojure-utils.scm        | 204 +++++++++++++++++++++++++++-
 5 files changed, 567 insertions(+), 2 deletions(-)
 create mode 100644 guix/build-system/clojure.scm
 create mode 100644 guix/build/clojure-build-system.scm

diff --git a/Makefile.am b/Makefile.am
index e2bc4d369..fce2956b4 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -101,6 +101,7 @@ MODULES =                                   \
   guix/build-system/android-ndk.scm            \
   guix/build-system/ant.scm                    \
   guix/build-system/cargo.scm                  \
+  guix/build-system/clojure.scm                        \
   guix/build-system/cmake.scm                  \
   guix/build-system/dub.scm                    \
   guix/build-system/emacs.scm                  \
@@ -138,6 +139,7 @@ MODULES =                                   \
   guix/build/download.scm                      \
   guix/build/download-nar.scm                  \
   guix/build/cargo-build-system.scm            \
+  guix/build/clojure-build-system.scm          \
   guix/build/cmake-build-system.scm            \
   guix/build/dub-build-system.scm              \
   guix/build/emacs-build-system.scm            \
diff --git a/doc/guix.texi b/doc/guix.texi
index b41af61f1..8b353e166 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -51,6 +51,7 @@ Copyright @copyright{} 2018 Mike address@hidden
 Copyright @copyright{} 2018 Pierre-Antoine address@hidden
 Copyright @copyright{} 2018 Gábor address@hidden
 Copyright @copyright{} 2018 Florian address@hidden
+Copyright @copyright{} 2018 Alex address@hidden
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -4384,6 +4385,63 @@ The @code{install} phase installs the binaries, and it 
also installs the
 source code and @file{Cargo.toml} file.
 @end defvr
 
address@hidden Clojure (programming language)
address@hidden simple Clojure build system
address@hidden {Scheme Variable} clojure-build-system
+This variable is exported by @code{(guix build-system clojure)}.  It implements
+a simple build procedure for @uref{https://clojure.org/, Clojure} packages
+using plain old @code{compile} in Clojure.  Cross-compilation is not supported
+yet.
+
+It adds @code{clojure}, @code{icedtea} and @code{zip} to the set of inputs.
+Different packages can be specified with the @code{#:clojure}, @code{#:jdk} and
address@hidden:zip} parameters, respectively.
+
+A list of source directories, test directories and jar names can be specified
+with the @code{#:source-dirs}, @code{#:test-dirs} and @code{#:jar-names}
+parameters, respectively.  Compile directory and main class can be specified
+with the @code{#:compile-dir} and @code{#:main-class} parameters, respectively.
+Other parameters are documented below.
+
+This build system is an extension of @var{ant-build-system}, but with the
+following phases changed:
+
address@hidden @code
+
address@hidden build
+This phase calls @code{compile} in Clojure to compile source files and runs
address@hidden to create jars from both source files and compiled files
+according to the include list and exclude list specified in
address@hidden:aot-include} and @code{#:aot-exclude}, respectively.  The 
exclude list
+has priority over the include list.  These lists consist of symbols
+representing Clojure libraries or the special keyword @code{#:all} representing
+all Clojure libraries found under the source directories.  The parameter
address@hidden:omit-source?} decides if source should be included into the jars.
+
address@hidden check
+This phase runs tests according to the include list and exclude list specified
+in @code{#:test-include} and @code{#:test-exclude}, respectively.  Their
+meanings are analogous to that of @code{#:aot-include} and
address@hidden:aot-exclude}, except that the special keyword @code{#:all} now
+stands for all Clojure libraries found under the test directories.  The
+parameter @code{#:tests?} decides if tests should be run.
+
address@hidden install
+This phase installs all jars built previously.
address@hidden table
+
+Apart from the above, this build system also contains an additional phase:
+
address@hidden @code
+
address@hidden install-doc
+This phase installs all top-level files with base name matching
address@hidden  A different regex can be specified with the
address@hidden:doc-regex} parameter.  All files (recursively) inside the 
documentation
+directories specified in @code{#:doc-dirs} are installed as well.
address@hidden table
address@hidden defvr
+
 @defvr {Scheme Variable} cmake-build-system
 This variable is exported by @code{(guix build-system cmake)}.  It
 implements the build procedure for packages using the
diff --git a/guix/build-system/clojure.scm b/guix/build-system/clojure.scm
new file mode 100644
index 000000000..5a91bcba0
--- /dev/null
+++ b/guix/build-system/clojure.scm
@@ -0,0 +1,195 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Alex Vong <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build-system clojure)
+  #:use-module (guix build clojure-utils)
+  #:use-module (guix build-system)
+  #:use-module (guix build-system ant)
+  #:use-module ((guix build-system gnu)
+                #:select (standard-packages)
+                #:prefix gnu:)
+
+  #:use-module (guix derivations)
+  #:use-module (guix packages)
+  #:use-module ((guix search-paths)
+                #:select
+                ((search-path-specification->sexp . search-path-spec->sexp)))
+  #:use-module (guix utils)
+
+  #:use-module (ice-9 match)
+  #:export (%clojure-build-system-modules
+            clojure-build
+            clojure-build-system))
+
+;; Commentary:
+;;
+;; Standard build procedure for Clojure packages.
+;;
+;; Code:
+
+(define-with-docs %clojure-build-system-modules
+  "Build-side modules imported and used by default."
+  `((guix build clojure-build-system)
+    (guix build clojure-utils)
+    (guix build guile-build-system)
+    ,@%ant-build-system-modules))
+
+(define-with-docs %default-clojure
+  "The default Clojure package."
+  (delay (@* (gnu packages lisp) clojure)))
+
+(define-with-docs %default-jdk
+  "The default JDK package."
+  (delay (@* (gnu packages java) icedtea)))
+
+(define-with-docs %default-zip
+  "The default ZIP package."
+  (delay (@* (gnu packages compression) zip)))
+
+(define* (lower name
+                #:key
+                source target
+                inputs native-inputs
+                (clojure (force %default-clojure))
+                (jdk (force %default-jdk))
+                (zip (force %default-zip))
+                outputs system
+                #:allow-other-keys
+                #:rest arguments)
+  "Return a bag for NAME."
+  (let ((private-keywords '(#:source #:target
+                            #:inputs #:native-inputs
+                            #:clojure #:jdk #:zip)))
+
+    (if target
+        (error "No cross-compilation for clojure-build-system yet: LOWER"
+               target) ; FIXME
+        (bag (name name)
+             (system system)
+             (host-inputs `(,@(if source
+                                  `(("source" ,source))
+                                  '())
+                            ,@inputs
+                            ,@(gnu:standard-packages)))
+             (build-inputs `(("clojure" ,clojure)
+                             ("jdk" ,jdk "jdk")
+                             ("zip" ,zip)
+                             ,@native-inputs))
+             (outputs outputs)
+             (build clojure-build)
+             (arguments (strip-keyword-arguments private-keywords
+                                                 arguments))))))
+
+(define-with-docs source->output-path
+  "Convert source input to output path."
+  (match-lambda
+    (((? derivation? source))
+     (derivation->output-path source))
+    ((source)
+     source)
+    (source
+     source)))
+
+(define-with-docs maybe-guile->guile
+  "Find the right guile."
+  (match-lambda
+    ((and maybe-guile (? package?))
+     maybe-guile)
+    (#f ; default
+     (@* (gnu packages commencement) guile-final))))
+
+(define* (clojure-build store name inputs
+                        #:key
+                        (source-dirs `',%source-dirs)
+                        (test-dirs `',%test-dirs)
+                        (compile-dir %compile-dir)
+
+                        (jar-names `',(package-name->jar-names name))
+                        (main-class %main-class)
+                        (omit-source? %omit-source?)
+
+                        (aot-include `',%aot-include)
+                        (aot-exclude `',%aot-exclude)
+
+                        doc-dirs ; no sensible default
+                        (doc-regex %doc-regex)
+
+                        (tests? %tests?)
+                        (test-include `',%test-include)
+                        (test-exclude `',%test-exclude)
+
+                        (phases '(@ (guix build clojure-build-system)
+                                    %standard-phases))
+                        (outputs '("out"))
+                        (search-paths '())
+                        (system (%current-system))
+                        (guile #f)
+
+                        (imported-modules %clojure-build-system-modules)
+                        (modules %clojure-build-system-modules))
+  "Build SOURCE with INPUTS."
+  (let ((builder `(begin
+                    (use-modules ,@modules)
+                    (clojure-build #:name ,name
+                                   #:source ,(source->output-path
+                                              (assoc-ref inputs "source"))
+
+                                   #:source-dirs ,source-dirs
+                                   #:test-dirs ,test-dirs
+                                   #:compile-dir ,compile-dir
+
+                                   #:jar-names ,jar-names
+                                   #:main-class ,main-class
+                                   #:omit-source? ,omit-source?
+
+                                   #:aot-include ,aot-include
+                                   #:aot-exclude ,aot-exclude
+
+                                   #:doc-dirs ,doc-dirs
+                                   #:doc-regex ,doc-regex
+
+                                   #:tests? ,tests?
+                                   #:test-include ,test-include
+                                   #:test-exclude ,test-exclude
+
+                                   #:phases ,phases
+                                   #:outputs %outputs
+                                   #:search-paths ',(map search-path-spec->sexp
+                                                         search-paths)
+                                   #:system ,system
+                                   #:inputs %build-inputs)))
+
+        (guile-for-build (package-derivation store
+                                             (maybe-guile->guile guile)
+                                             system
+                                             #:graft? #f)))
+
+    (build-expression->derivation store name builder
+                                  #:inputs inputs
+                                  #:system system
+                                  #:modules imported-modules
+                                  #:outputs outputs
+                                  #:guile-for-build guile-for-build)))
+
+(define clojure-build-system
+  (build-system
+    (name 'clojure)
+    (description "Simple Clojure build system using plain old 'compile'")
+    (lower lower)))
+
+;;; clojure.scm ends here
diff --git a/guix/build/clojure-build-system.scm 
b/guix/build/clojure-build-system.scm
new file mode 100644
index 000000000..d8f7c89f8
--- /dev/null
+++ b/guix/build/clojure-build-system.scm
@@ -0,0 +1,110 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Alex Vong <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build clojure-build-system)
+  #:use-module ((guix build ant-build-system)
+                #:select ((%standard-phases . address@hidden)
+                          ant-build))
+  #:use-module (guix build clojure-utils)
+  #:use-module (guix build java-utils)
+  #:use-module (guix build utils)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:export (%standard-phases
+            clojure-build))
+
+;; Commentary:
+;;
+;; Builder-side code of the standard build procedure for Clojure packages.
+;;
+;; Code:
+
+(define* (build #:key
+                source-dirs compile-dir
+                jar-names main-class omit-source?
+                aot-include aot-exclude
+                #:allow-other-keys)
+  "Standard 'build' phase for clojure-build-system."
+  (let* ((libs (append-map find-clojure-libs source-dirs))
+         (libs* (include-list\exclude-list aot-include
+                                           aot-exclude
+                                           #:all-list libs)))
+    (mkdir-p compile-dir)
+    (eval-with-clojure `(run! compile ',libs*)
+                       source-dirs)
+    (let ((source-dir-files-alist (map (lambda (dir)
+                                         (cons dir (find-files* dir)))
+                                       source-dirs))
+          ;; workaround transitive compilation in Clojure
+          (classes (filter (lambda (class)
+                             (any (cut compiled-from? class <>)
+                                  libs*))
+                           (find-files* compile-dir))))
+      (for-each (cut create-jar <> (cons (cons compile-dir classes)
+                                         (if omit-source?
+                                             '()
+                                             source-dir-files-alist))
+                     #:main-class main-class)
+                jar-names)
+      #t)))
+
+(define* (check #:key
+                test-dirs
+                jar-names
+                tests? test-include test-exclude
+                #:allow-other-keys)
+  "Standard 'check' phase for clojure-build-system.  Note that TEST-EXCLUDE has
+priority over TEST-INCLUDE."
+  (if tests?
+      (let* ((libs (append-map find-clojure-libs test-dirs))
+             (libs* (include-list\exclude-list test-include
+                                               test-exclude
+                                               #:all-list libs)))
+        (for-each (lambda (jar)
+                    (eval-with-clojure `(do (apply require
+                                                   '(clojure.test ,@libs*))
+                                            (apply clojure.test/run-tests
+                                                   ',libs*))
+                                       (cons jar test-dirs)))
+                  jar-names)))
+  #t)
+
+(define-with-docs install
+  "Standard 'install' phase for clojure-build-system."
+  (install-jars "./"))
+
+(define-with-docs %standard-phases
+  "Standard build phases for clojure-build-system."
+  (modify-phases address@hidden
+    (replace 'build build)
+    (replace 'check check)
+    (replace 'install install)
+    (add-after 'install-license-files 'install-doc install-doc)))
+
+(define* (clojure-build #:key
+                        inputs
+                        (phases %standard-phases)
+                        #:allow-other-keys
+                        #:rest args)
+  "Build the given Clojure package, applying all of PHASES in order."
+  (apply ant-build
+         #:inputs inputs
+         #:phases phases
+         args))
+
+;;; clojure-build-system.scm ends here
diff --git a/guix/build/clojure-utils.scm b/guix/build/clojure-utils.scm
index 713dff2d8..027777b4d 100644
--- a/guix/build/clojure-utils.scm
+++ b/guix/build/clojure-utils.scm
@@ -19,12 +19,48 @@
 (define-module (guix build clojure-utils)
   #:use-module (guix build utils)
   #:use-module (ice-9 ftw)
+  #:use-module (ice-9 match)
   #:use-module (ice-9 regex)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-8)
   #:use-module (srfi srfi-26)
-  #:export (%clojure-regex
+  #:export (@*
+            @@*
             define-with-docs
-            install-doc))
+
+            %doc-regex
+            install-doc
+
+            %source-dirs
+            %test-dirs
+            %compile-dir
+            package-name->jar-names
+            %main-class
+            %omit-source?
+            %aot-include
+            %aot-exclude
+            %tests?
+            %test-include
+            %test-exclude
+
+            %clojure-regex
+            canonicalize-relative-path
+            find-files*
+            file-sans-extension
+            relative-path->clojure-lib-string
+            find-clojure-libs
+            compiled-from?
+            include-list\exclude-list
+            eval-with-clojure
+            create-jar))
+
+(define-syntax-rule (@* module name)
+  "Like (@ MODULE NAME), but resolves at run time."
+  (module-ref (resolve-interface 'module) 'name))
+
+(define-syntax-rule (@@* module name)
+  "Like (@@ MODULE NAME), but resolves at run time."
+  (module-ref (resolve-module 'module) 'name))
 
 (define-syntax-rule (define-with-docs name docs val)
   "Create top-level variable named NAME with doc string DOCS and value VAL."
@@ -63,3 +99,167 @@ DOC-REGEX can be compiled or uncompiled."
     (for-each (cut copy-recursively <> dest-dir)
               doc-dirs)
     #t))
+
+(define-with-docs %source-dirs
+  "A default list of source directories."
+  '("src/"))
+
+(define-with-docs %test-dirs
+  "A default list of test directories."
+  '("test/"))
+
+(define-with-docs %compile-dir
+  "Default directory for holding class files."
+  "classes/")
+
+(define (package-name->jar-names name)
+  "Given NAME, a package name like \"foo-0.9.1b\",
+return the list of default jar names: (\"foo-0.9.1b.jar\" \"foo.jar\")."
+  (map (cut string-append <> ".jar")
+       (list name
+             (receive (base-name _)
+                 (package-name->name+version name)
+               base-name))))
+
+(define-with-docs %main-class
+  "Default name for main class.  It should be a symbol or #f."
+  #f)
+
+(define-with-docs %omit-source?
+  "Include source in jars by default."
+  #f)
+
+(define-with-docs %aot-include
+  "A default list of symbols deciding what to compile.  Note that the exclude
+list has priority over the include list.  The special keyword #:all represents
+all libraries found under the source directories."
+  '(#:all))
+
+(define-with-docs %aot-exclude
+  "A default list of symbols deciding what not to compile.
+See the doc string of '%aot-include' for more details."
+  '())
+
+(define-with-docs %tests?
+  "Enable tests by default."
+  #t)
+
+(define-with-docs %test-include
+  "A default list of symbols deciding what tests to include.  Note that the
+exclude list has priority over the include list.  The special keyword #:all
+represents all tests found under the test directories."
+  '(#:all))
+
+(define-with-docs %test-exclude
+  "A default list of symbols deciding what tests to exclude.
+See the doc string of '%test-include' for more details."
+  '())
+
+(define-with-docs %clojure-regex
+  "Default regex for matching the base name of clojure source files."
+  "\\.cljc?$")
+
+(define-with-docs canonicalize-relative-path
+  "Like 'canonicalize-path', but for relative paths.
+Canonicalizations requiring the path to exist are omitted."
+  (let ((remove.. (lambda (ls)
+                    (fold-right (match-lambda*
+                                  (((and comp (not "..")) (".." comps ...))
+                                   comps)
+                                  ((comp (comps ...))
+                                   (cons comp comps)))
+                                '()
+                                ls))))
+    (compose (match-lambda
+               (() ".")
+               (ls (string-join ls "/")))
+             remove..
+             (cut remove (cut member <> '("" ".")) <>)
+             (cut string-split <> #\/))))
+
+(define (find-files* base-dir . args)
+  "Similar to 'find-files', but with BASE-DIR stripped and result
+canonicalized."
+  (map canonicalize-relative-path
+       (with-directory-excursion base-dir
+         (apply find-files "./" args))))
+
+;;; FIXME: should be moved to (guix build utils)
+(define-with-docs file-sans-extension
+  "Strip extension from path, if any."
+  (@@ (guix build guile-build-system)
+      file-sans-extension))
+
+(define (relative-path->clojure-lib-string path)
+  "Convert PATH to a clojure library string."
+  (string-map (match-lambda
+                (#\/ #\.)
+                (#\_ #\-)
+                (chr chr))
+              (file-sans-extension path)))
+
+(define* (find-clojure-libs base-dir
+                            #:key (clojure-regex %clojure-regex))
+  "Return the list of clojure libraries found under BASE-DIR.
+
+CLOJURE-REGEX can be compiled or uncompiled."
+  (map (compose string->symbol
+                relative-path->clojure-lib-string)
+       (find-files* base-dir clojure-regex)))
+
+(define (compiled-from? class lib)
+  "Given class file CLASS and clojure library symbol LIB, decide if CLASS
+results from compiling LIB."
+  (string-prefix? (symbol->string lib)
+                  (relative-path->clojure-lib-string class)))
+
+(define* (include-list\exclude-list include-list exclude-list
+                                    #:key all-list)
+  "Given INCLUDE-LIST and EXCLUDE-LIST, replace all occurences of #:all by
+slicing ALL-LIST into them and compute their list difference."
+  (define (replace-#:all ls all-ls)
+    (append-map (match-lambda
+                  (#:all all-ls)
+                  (x (list x)))
+                ls))
+  (let ((include-list* (replace-#:all include-list all-list))
+        (exclude-list* (replace-#:all exclude-list all-list)))
+    (lset-difference equal? include-list* exclude-list*)))
+
+(define (eval-with-clojure expr extra-paths)
+  "Evaluate EXPR with clojure.
+
+EXPR must be a s-expression writable by guile and readable by clojure.
+For examples, '(require '[clojure.string]) will not work,
+because the guile writer converts brackets to parentheses.
+
+EXTRA-PATHS is a list of paths which will be appended to $CLASSPATH."
+  (let* ((classpath (getenv "CLASSPATH"))
+         (classpath* (string-join (cons classpath extra-paths) ":")))
+    (invoke "java"
+            "-classpath" classpath*
+            "clojure.main"
+            "--eval" (object->string expr))))
+
+(define* (create-jar output-jar dir-files-alist
+                     #:key
+                     (verbose? #t)
+                     (compress? #f)
+                     (main-class %main-class))
+  "Given DIR-FILES-ALIST, an alist of the form: ((DIR . FILES) ...)
+Create jar named OUTPUT-JAR from FILES with DIR stripped."
+  (let ((grouped-options (string-append "c"
+                                        (if verbose? "v" "")
+                                        "f"
+                                        (if compress? "" "0")
+                                        (if main-class "e" ""))))
+    (apply invoke `("jar"
+                    ,grouped-options
+                    ,output-jar
+                    ,@(if main-class (list (symbol->string main-class)) '())
+                    ,@(append-map (match-lambda
+                                    ((dir . files)
+                                     (append-map (lambda (file)
+                                                   `("-C" ,dir ,file))
+                                                 files)))
+                                  dir-files-alist)))))
-- 
2.19.1

signature.asc
Description: PGP signature