Re: Something is wrong with the translation project

[Paul Eggert]

Akim Demaille <address@hidden> writes:

> Something is wrong with the translation project.

Yes, it does seem to be on some sort of hiatus / overloaded state.
This affects Bison and other projects.

I asked the Translation Project about this on June 7
<http://lists.gnu.org/archive/html/bug-bison/2006-06/msg00008.html>
but haven't received a response.

For lack of a better alternative I installed the following screen
scraper (Ouch!) to get the latest translations into Bison.  I suppose
I'll have to go round to all my other projects and do this.  (Double ouch!)

2006-06-19  Paul Eggert  <address@hidden>

        The Translation Project changed its way of publishing translations
        to maintainers.  I haven't received any responses to my request
        for supporting the old way, or for documenting the new way.  I
        have modified 'bootstrap' to use screen scraping
        (in this case, HTML scraping).  This is unreliable and inelegant,
        but I don't see any better way.  Yuck.
        * bootstrap (TP_URL, WGET_COMMAND): New vars.
        (get_translations): New function, which uses HTML scraping to
        deduce locations of latest translations.
        Use this function to grab both bison and bison-runtime .po files.
        Don't bother priming the pump for the runtime-po domain any more,
        as it's now translated better than bison is.

Index: bootstrap
===================================================================
RCS file: /cvsroot/bison/bison/bootstrap,v
retrieving revision 1.35
diff -p -u -r1.35 bootstrap
--- bootstrap   25 Apr 2006 21:02:36 -0000      1.35
+++ bootstrap   19 Jun 2006 20:30:42 -0000
@@ -23,6 +23,9 @@
 
 package=bison
 
+# Translation Project URL, for the registry of all projects.
+TP_URL='http://www.iro.umontreal.ca/translation/registry.cgi?domain='
+
 # Ensure file names are sorted consistently across platforms;
 # e.g., m4/ulonglong_gl.m4 should follow m4/ulonglong.m4.
 # Also, ensure diagnostics are in English, e.g., "wget --help" below.
@@ -168,6 +171,34 @@ sed '
 
 # Get translations.
 
+get_translations() {
+  subdir=$1
+  domain=$2
+
+  echo "$0: getting translations into $subdir for $domain..."
+  (cd $subdir && rm -f dummy `ls | sed -n '/\.gmo$/p; /\.po/p'`) &&
+
+  $WGET_COMMAND -O "$subdir/$domain.html" "$TP_URL$domain" &&
+
+  sed -n 
's|.*"http://[^"]*/translation/teams/PO/\([^/"]*\)/'"$domain"'-\([^/"]*\)\.[^."]*\.po".*|\1.\2|p'
 <"$subdir/$domain.html" |
+  sort -k 1,1 -k 2,2n -k2,2 -k3,3n -k3,3 -k4,4n -k4,4 -k5,5n -k5.5 |
+  awk -F. '
+    { if (lang && $1 != lang) print lang, ver }
+    { lang = $1; ver = substr($0, index($0, ".") + 1) }
+    END { if (lang) print lang, ver }
+  ' | awk -v domain="$domain" -v subdir="$subdir" '
+    {
+      lang = $1
+      ver = $2
+      urlfmt = ""
+      printf "$WGET_COMMAND -O %s/%s.po 
'http://www.iro.umontreal.ca/translation/teams/PO/%s/%s-%s.%s.po' &&\n", 
subdir, lang, lang, domain, ver, lang
+    }
+    END { print ":" }
+  ' | sh &&
+  ls "$subdir"/*.po | sed 's|.*/||; s|\.po$||' >"$subdir/LINGUAS" &&
+  rm "$subdir/$domain.html"
+}
+
 case $SKIP_PO in
 '')
   case `wget --help` in
@@ -179,51 +210,14 @@ case $SKIP_PO in
     no_cache='';;
   esac
 
-  echo "$0: getting translations into po (please ignore the robots.txt ERROR 
404)..."
-  (cd po &&
-   rm -f dummy `ls | sed -n '/\.gmo$/p; /\.po/p'` &&
-   wget -nv -nd -r -l 1 -A .po $no_cache \
-     http://www.iro.umontreal.ca/translation/maint/$package/ &&
-   ls *.po | sed 's/\.po$//' >LINGUAS
-  ) || exit
+  WGET_COMMAND="wget -nv $no_cache"
+  export WGET_COMMAND
+
+  get_translations po $package || exit
 
   case $package in
   bison)
-    echo "$0: getting translations into po (please ignore the robots.txt ERROR 
404)..."
-    (cd runtime-po &&
-     rm -f dummy `ls | sed -n '/\.gmo$/p; /\.po$/p'` &&
-     wget -nv -nd -r -l 1 -A .po $no_cache \
-       http://www.iro.umontreal.ca/translation/maint/$package-runtime/ &&
-
-     # For translations that have not yet been upgraded to the new
-     # runtime-po domain, prime the pump by extracting the relevant
-     # strings from the obsolete translations.
-     # This code can be removed once the bison-runtime domain
-     # has been translated by each team.
-     for po in ../po/*.po; do
-       test -f "$po" || continue
-       runpo=`basename $po`
-       test -f $runpo || {
-        msggrep -K \
-            -e 'memory exhausted' \
-            -e 'syntax error' \
-            $po 2>/dev/null |
-          sed '
-            s/^#~ //
-            /^msgid "syntax error; also memory exhausted"$/,/^$/d
-            /^$/,${ /^#/d; }
-          ' >$runpo-
-        if cmp -s $runpo- $runpo; then
-          rm $runpo-
-        else
-          mv $runpo- $runpo
-        fi
-        test -s $runpo || rm -f $runpo
-       } || exit
-     done &&
-
-     ls *.po | sed 's/\.po$//' >LINGUAS
-    ) || exit
+    get_translations runtime-po $package-runtime || exit
   esac;;
 esac