[Savannah-cvs] [142] Internals: expand vcs cronjobs information

[Assaf Gordon]

Revision: 142
          
http://svn.sv.gnu.org/viewvc/?view=rev&root=administration&revision=142
Author:   agn
Date:     2015-02-16 01:45:36 +0000 (Mon, 16 Feb 2015)
Log Message:
-----------
Internals: expand vcs cronjobs information

Modified Paths:
--------------
    trunk/sviki/SavannahDatabaseSchema.mdwn
    trunk/sviki/SavannahInternals.mdwn

Modified: trunk/sviki/SavannahDatabaseSchema.mdwn
===================================================================
--- trunk/sviki/SavannahDatabaseSchema.mdwn     2015-02-15 23:24:18 UTC (rev 
141)
+++ trunk/sviki/SavannahDatabaseSchema.mdwn     2015-02-16 01:45:36 UTC (rev 
142)
@@ -89,6 +89,11 @@
     +---------+---------------------+-------------------------+
     1 row in set (0.02 sec)
 
+TODO:
+
+* explain `user.status='SQD'` used for project "squads" (per-project
+  mailing lists. used with `sv_aliases` on `vcs`.
+
 ## `user_group` table
 
 Project membership of user `rms`, same as shown at

Modified: trunk/sviki/SavannahInternals.mdwn
===================================================================
--- trunk/sviki/SavannahInternals.mdwn  2015-02-15 23:24:18 UTC (rev 141)
+++ trunk/sviki/SavannahInternals.mdwn  2015-02-16 01:45:36 UTC (rev 142)
@@ -44,7 +44,7 @@
 scripts is available at:
 <http://git.savannah.gnu.org/cgit/administration/savane.git>.
 N.B. That's the `administration` project/group, and the
-git repository named `savane' within it.
+git repository named `savane` within it.
 
 To experiment with running a local copy of Savane see [[RunningSavaneLocally]]
 and [[SavaneInABox]].
@@ -306,8 +306,6 @@
 
 ### Cron job on vcs
 
-FIXME: work-in-progress
-
 Stored in `vcs:/etc/cron.d/sv`:
 
     # remake user list for outgoing mail.
@@ -322,6 +320,117 @@
     # remake list of groups for the vcs.
     35,05 * * * *      root    nice -n 11 sv_groups --cron --only-cvs 
--only-homepage --only-svn --only-git --only-hg --only-bzr
 
+* 
[/backend/mail/sv_aliases.in](http://git.savannah.gnu.org/cgit/administration/savane.git/tree/backend/mail/sv_aliases.in)
+  (installed in `vcs:/usr/local/bin/sv_aliases`) - This script prepares an
+  `alias`-type file for for exim4 mail server, used when sending commit
+  updates to mailing lists.
+    * The script queries the MySQL database, extracting user names and
+      emails for all savannah users. Additionally, emails for 'squads'
+      (per-project mailing groups) are queried. The results are saved in
+      `vcs:/etc/email-addresses`.
+    * `vcs:/etc/email-addresses` is symlinked to the standard
+      `vcs:/etc/exim4/email-addresses` file. Example:
+
+          agn: address@hidden
+          karl: address@hidden
+
+
+    * The `sed` command in the cronjob file replaces each user's email
+      with address@hidden, and saves the results in
+      `vcs:/etc/email-addresses_SENDER`. Example:
+
+          agn: address@hidden
+          karl: address@hidden
+
+    * FIXME: where is `/etc/email-addresses_SENDER` mentioned/used?
+      in an exim4 config file?
+
+* `refresh-project-list.sh` - this script updates the
+  `vcs:/srv/git/project-list` file which is used by gitweb CGI application.
+    * FIXME: file is `vcs:/usr/src/infra/git/refresh-project-list.sh`, and
+      is not in the `savane` repository. Add it?
+    * gitweb configuration file is `vcs:/etc/gitweb.conf`.
+    * Implemetation detail: the script does not re-generate the
+      `project-list` from scratch. Instead, it scans `/srv/git` for new git
+      repositories, and only adds them to the file if they aren't already
+      listed. This allows manually adding information to the `project-list`
+      file, ensuring it will not be overwritten. The manually added
+      information is the owner name/email for some repositories. Example:
+
+          $ cat /srv/git/project-list
+          ...
+          administration.git Savannah+Hackers
+          anubis.git address@hidden
+          autoconf.git address@hidden
+          automake.git address@hidden
+          autostrap.git Sylvain+Beucler
+          coreutils.git Jim+Meyering
+          ...
+
+      These names (e.g. `Jim Meyering`) will appear
+      in the *owner* field when viewing the gitweb page of the project
+      (example: <http://git.savannah.gnu.org/gitweb/?p=coreutils.git>).
+
+* `sv_cgit.pl` - this script updates the `vcs:/srv/git/cgitrepos` file
+  which is used by the cgit CGI application.
+    * FIXME: file is `vcs:/usr/src/infra/git/sv_cgit.pl` and is not in the
+      `savane` repository. Add it?
+    * cgit configuration file is `vcs:/etc/cgitrc`.
+    * Implementation detail: the script reads the `/srv/git/project-list`
+      file (generated by `refresh-project-list.sh`, above), and for each
+      mentioned project, writes an entry in `cgitrepos`. Example:
+
+
+          $ cat /srv/git/cgitrepos
+          ...
+          repo.group=GNU Core Utilities
+          repo.url=coreutils.git
+          repo.path=/srv/git/coreutils.git
+          repo.desc=GNU coreutils
+          repo.readme=README.html
+          repo.owner=Jim Meyering
+          ...
+
+* `generate_log_accum.pl` - This script updates the CVS hooks in savannah
+  repositories.
+    * FIXME: file is `vcs:/usr/src/infra/cvs/generate_log_accum.pl` and
+      is not in the `savane` repository. Add it?
+    * The script queries the database (`groups` table) for project using
+      cvs repositories, and projects using homepages (which use the cvsweb
+      repositories).
+    * For each cvs repository, it creates/updates the
+      `<project>/CVSROOT/{loginfo,commitinfo}` files.
+    * Example of `commitinfo` for source-code (not web) CVS repository:
+
+          $ cat /sources/pspp/CVSROOT/commitinfo
+          #<savane>
+          DEFAULT /usr/local/bin/commit_prep.pl -T pspp_70 %p
+          #</savane>
+          ALL $CVSROOT/CVSROOT/no-commits %{s}
+
+    * Example of `loginfo` for source-code (not web) CVS repository:
+
+          $ cat /sources/pspp/CVSROOT/loginfo
+          #<savane>
+          DEFAULT /usr/local/bin/log_accum.pl --mail-to address@hidden 
--send-diff --file-prefix pspp_70 --config /etc/log_accum-sources.config %p 
%{sVv}
+          #</savane>
+
+    * Example of `loginfo` for web CVS repository:
+
+          $ cat /srv/cvs/web/datamash/CVSROOT/loginfo
+          #<savane>
+          ALL echo 'Triggering webpages update...'; cat > /dev/null; curl 
http://www.gnu.org/new-savannah-project/new.py -s -F type=gnu -F 
project=`basename %r`
+          #</savane>
+
+* `sv_groups` - This script creates new repositories for projects in the
+  savannah database
+    * reminder: when a new project is approved/created on
+      savannah, or when a project administrator selects a new repository
+      feature the `frontend` php code only updates the database. this
+      script picks up the changes and executes them).
+    * file is 
[/backend/accounts/sv_groups.in](http://git.savannah.gnu.org/cgit/administration/savane.git/tree/backend/accounts/sv_groups.in).
+
+
 ### Cron jobs on download
 
 FIXME: work-in-progress
@@ -334,7 +443,9 @@
 
 FIXME: work-in-progress
 
-stored in `frontend:/etc/cron.d/sv_{export,list_groups}` - what are these for?
+* `frontend:/etc/cron.d/sv_export` - unused?
+* `frontend:/etc/cron.daily/sv_list_groups` - unfinished. publish
+  authoritative list of gnu/nongnu projects on savannah.
 
 FIXME: where's the script which creates lists on `lists.gnu.org` (
 calls `sv_mailman`)?