[taler-docs] branch master updated: DD43

[gnunet]

This is an automated email from the git hooks/post-receive script.

dold pushed a commit to branch master
in repository docs.

The following commit(s) were added to refs/heads/master by this push:
     new 71d6cb6  DD43
71d6cb6 is described below

commit 71d6cb62b406756afdc9e4df1a6b6dcf01f4fb85
Author: Florian Dold <florian@dold.me>
AuthorDate: Thu May 4 20:11:50 2023 +0200

    DD43
---
 .../043-managing-prebuilt-artifacts.rst            | 101 +++++++++++++++++++++
 design-documents/index.rst                         |   1 +
 2 files changed, 102 insertions(+)

diff --git a/design-documents/043-managing-prebuilt-artifacts.rst 
b/design-documents/043-managing-prebuilt-artifacts.rst
new file mode 100644
index 0000000..0082139
--- /dev/null
+++ b/design-documents/043-managing-prebuilt-artifacts.rst
@@ -0,0 +1,101 @@
+DD 43: Managing Prebuilt Artifacts
+##################################
+
+Summary
+=======
+
+This design document defines how the GNU Taler project manages prebuilt
+artifacts that are used as source-level dependencies in source code
+repositories.
+
+Motivation
+==========
+
+Some repositories have source-level dependencies on the build results of other
+repositories.  While it is possible to build these dependencies from scratch,
+we often want to avoid that extra step in order to make building the software
+easier and faster.
+
+Examples are:
+
+* man-pages built via Sphinx, produced in ``docs.git`` and consumed in 
``exchange.git``.
+* SPAs used by the exchange, merchant libeufin.
+* (formerly): The ``taler-wallet-core-embedded.js`` file used
+  by the Android repo.
+
+Requirements
+============
+
+* We are a free software project, users must always be able to
+  easily re-build the prebuilt artifacts for themselves.
+* We want to reduce reliance on third party infrastructure as much
+  as possible.
+* We want to keep our own infrastructure as slim as possible.
+* Builds must be reproducible and remain reproducible in the future.
+* Offline builds should be supported as well as possible.
+* Ideally, prebuilt artifacts can easily be cached / archived by third parties.
+
+Proposed Solution
+=================
+
+Instead of using a full-blown artifact management solution from the start, we
+make use of Git.  Since Git is not very good at managing binary artifacts,
+prebuilt files are not managed alongside the source code. Instead, we use
+(orphan-)branches) in a (possibly separate) repository to manage them.  This
+allows us to re-use Git authentication and Git commit signing.
+
+Due to a historical accident, prebuilt files are currently stored in the
+``prebuilt`` branch of the ``wallet-core.git`` repository.  This might change
+in the future.
+
+To enable forwards-compatibility, we are implementing the following approach:
+The ``taler.net`` HTTP server redirects requests in the form of
+``https://artifacts.taler.net/$component/$version/$file`` to
+``https://git.taler.net/$repo.git/tree/$version/$file?h=$branch``, where
+``$component`` determines ``$repo`` and ``$branch``.
+
+We are also introducing the new rule that a prebuilt artifact must be a single
+archive file (preferable ``.tar.gz.``).
+
+Repositories that produce artifacts *MUST* have phony Makefile helper targets
+to (a) create the prebuilt artifact (b) check out the prebuilt branch as a Git 
worktree, (c)
+move the artifact to the right place in the prebuilt branch and commit/push.
+
+Repositories that depend on prebuilt components must download them
+in their ``./bootstrap`` script.  After downloading, the bootstrap
+script *SHOULD* verify a checksum of the downloaded artifact.
+
+FIXME: Link to an example bootstrap file, once the approach from this document
+is fully implemented.
+
+Alternatives
+============
+
+* The prebuilt branch could be integrated into source repositories that
+  depend on it via a git submodule.  This has the following disadvantages:
+
+  1. Git submodules are difficult to handle for some developers. We've had 
issues
+     in the past where the prebuilt submodule wasn't integrated properly and 
would
+     lead to non-reproducible builds
+  2. Relying purely on the Git commit hash of the prebuilt branch
+     makes it very difficult to know which version of the prebuilt
+     artifact is being used.  It also makes it very difficult
+     to reliable find older versions of the artifact.
+  3. Relying on the commit hash instead of a version and artifact hash
+     couples the artifact to the Git commit.  That means that
+     we can never compact the commit history of the prebuilt
+     branch of move artifacts around.
+
+* Set up a full-blown artifact management solution like JFrog Artifactory.
+  That's costly and requires a lot of admin work.
+
+* Never rely on prebuilt files.  That slows down the build process
+  and in some cases requires exotic dependencies.
+
+Drawbacks
+=========
+
+Discussion / Q&A
+================
+
+(This should be filled in with results from discussions on mailing lists / 
personal communication.)
diff --git a/design-documents/index.rst b/design-documents/index.rst
index fd2ca7b..e191c0c 100644
--- a/design-documents/index.rst
+++ b/design-documents/index.rst
@@ -51,4 +51,5 @@ and protocol.
   040-distro-packaging.rst
   041-wallet-balance-amount-definitions.rst
   042-synthetic-wallet-errors.rst
+  043-managing-prebuilt-artifacts.rst
   999-template

-- 
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.