po / pot file integration: documentation

[Bruno Haible]

Hi Ralf,

Thank you a lot for the willingness to integrate the handling of PO and POT
files into Automake (first proposed in 2003:
<http://sources.redhat.com/ml/automake/2003-11/msg00047.html>).

In order to test our common understanding, and to prepare for the unit tests,
here is a proposed patch to the documentation. Feel free to put it on a
git branch.


2010-08-15  Bruno Haible  <address@hidden>

        Document the handling of POT and PO files.
        * doc/automake.texi (Public Macros): Document AM_POT_TOOLS.
        (Private Macros): Add AM_NLS.
        (Internationalization): New chapter.

--- doc/automake.texi.orig      Sun Aug 15 22:15:54 2010
+++ doc/automake.texi   Sun Aug 15 22:13:56 2010
@@ -105,6 +106,7 @@
 * Directories::                 Declaring subdirectories
 * Programs::                    Building programs and libraries
 * Other Objects::               Other derived objects
+* Internationalization::        Declaring translatable message catalogs
 * Other GNU Tools::             Other GNU Tools
 * Documentation::               Building documentation
 * Install::                     What gets installed
@@ -277,6 +279,32 @@
 
 * Built Sources Example::       Several ways to handle built sources.
 
+Internationalization
+
+* POT Files::                   Declaring a POT file
+* Parametrizing xgettext::      Declaring how a POT file gets created
+* Message catalog translations::Declaring the translations
+* Message catalog installation::Declaring how message catalogs get installed
+* Other POT files details::     Rarely used settings for POT files
+
+Parametrizing xgettext
+
+* A POT file's _SOURCES::            Declaring the source files.
+* A POT file's _COPYRIGHT_HOLDER::   The message catalogs' copyright holder
+* A POT file's _MSGID_BUGS_ADDRESS:: Allowing translators to report bugs
+* Additional xgettext options::      Fine tuning of xgettext
+* Multiple xgettext invocations::    Advanced uses of xgettext
+
+Message catalog installation
+
+* A POT file's _CATALOGFORMAT::      Formats of compiled message catalogs
+* A POT file's _LOCALE_CATEGORIES::  Using different locale categories
+
+Other POT files details
+
+* A POT file's _USE_MSGCTXT::        Declaring whether msgctxt is used
+* A POT file's _MSGMERGE_OPTIONS::   Specifying options for msgmerge
+
 Other GNU Tools
 
 * Emacs Lisp::                  Emacs Lisp
@@ -3861,6 +3889,19 @@
 @command{configure} to explicitly set the correct path (if you're sure
 you have an @command{emacs} that supports Emacs Lisp).
 
address@hidden AM_POT_TOOLS
address@hidden AM_POT_TOOLS
+
+The @code{AM_POT_TOOLS} macro performs tests for the @code{POTS} primary.
+See @pxref{Programming Languages, , Programming Languages, gettext,
+GNU gettext tools} for a list of programming languages that support
+localization through PO files.
+
+The @code{AM_POT_TOOLS} macro determines whether internationalization
+should be used.  If so, it sets the @code{USE_NLS} variable to @samp{yes},
+otherwise to @samp{no}.  It also determines the right values for Makefile
+variables used by the Makefile rules for the POT and PO files.
+
 @item AM_PROG_AS
 @acindex AM_PROG_AS
 @vindex CCAS
@@ -4030,6 +4071,11 @@
 @code{include} statements.  This macro is automatically invoked when
 needed; there should be no need to invoke it manually.
 
address@hidden AM_NLS
+This macro determines whether internationalization should be used.  If
+so, it sets the @code{USE_NLS} variable to @samp{yes}, otherwise to
address@hidden
+
 @item AM_PROG_INSTALL_STRIP
 This is used to find a version of @code{install} that can be used to
 strip a program at installation time.  This macro is automatically
@@ -7352,6 +7398,421 @@
 is converting @file{.h} files into @file{.c} files.
 
 
address@hidden Internationalization
address@hidden Internationalization
+
+An internationalized program is a program that can communicate with the
+user in his native language, provided that some translation work has been
+done.  An internationalized program can be localized to a certain
+language by translating the messages by which the program communicates
+with the user.  These message are grouped in files called ``message
+catalogs''.
+
+The process of localization consists of three steps:
+
address@hidden
address@hidden
+The developers send a file with English messages and (optionally)
+some translation hints for the translators.
address@hidden
+The translators translate these files to their respective language.
address@hidden
+The translators send back the translated message catalogs, and the
+developers integrate them in the package.
address@hidden enumerate
+
+Automake supports the message catalog format of GNU Gettext
+(@pxref{Top, , Introduction, gettext, GNU gettext tools}).  When the GNU
+Gettext tools are in use:
+
address@hidden
address@hidden
+The file that contains the English messages and translation hints is
+called a POT file (Portable Object Template) and usually named
address@hidden@var{domain}.pot}.  The @var{domain} is a identifier for the
+translation domain.  Different packages must use different @var{domain}s.
+Therefore, usually, the @var{domain} is the same as the package name.
+
address@hidden
+The file that contains the translation produced by a translation team
+is called a PO file (Portable Object) and usually named
address@hidden@address@hidden or
address@hidden@address@hidden@var{CC}.po}.  Here @var{ll} is the
+language code identifier according to ISO 639, and the optional @var{CC}
+is the country code identifier according to ISO 3166.  The optional
address@hidden is only needed to distinguish different dialects of the same
+language, such as @code{pt_BR} (Brazilian Portuguese), which is slightly
+different from @code{pt} (Portuguese).
address@hidden itemize
+
address@hidden
+* POT Files::                   Declaring a POT file
+* Parametrizing xgettext::      Declaring how a POT file gets created
+* Message catalog translations::Declaring the translations
+* Message catalog installation::Declaring how message catalogs get installed
+* Other POT files details::     Rarely used settings for POT files
address@hidden menu
+
address@hidden POT Files
address@hidden POT Files
+
+In Automake, a message catalog template (POT file) is a primary.  The
+Makefile rules generated by Automake also handle all the PO files that
+belong to the POT file.
+
+For execution, the PO files get compiled to ``compiled message
+catalogs''.  These a usally @code{.mo} files (MO = Machine Object).
+They get installed in an appropriate subdirectory of @code{$(localedir)}.
+
+The declaration of an installable message catalog domain therefore looks
+like this:
+
address@hidden
+locale_POTS = po/maude.pot
address@hidden smallexample
+
+This declaration causes Automake to emit Makefile rules that create
+the @file{po/maude.pot} file from some of the sources, and that manage
+translated message catalogs @file{po/address@hidden@var{CC}.po} in the
+same directory.
+
address@hidden AM_POT_TOOLS
+When you use the @code{POTS} primary, you need to invoke
address@hidden directly or indirectly from @file{configure.ac}.
+
address@hidden Parametrizing xgettext
address@hidden Parametrizing xgettext
+
+The POT file gets created through an invocation of the @code{xgettext}
+program
+(@pxref{xgettext Invocation, , xgettext, gettext, GNU gettext tools}).
+
address@hidden
+* A POT file's _SOURCES::            Declaring the source files.
+* A POT file's _COPYRIGHT_HOLDER::   The message catalogs' copyright holder
+* A POT file's _MSGID_BUGS_ADDRESS:: Allowing translators to report bugs
+* Additional xgettext options::      Fine tuning of xgettext
+* Multiple xgettext invocations::    Advanced uses of xgettext
address@hidden menu
+
address@hidden A POT file's _SOURCES
address@hidden A POT file's _SOURCES
+
address@hidden scans a set of source files.  The default set of sources
+is the union of all _SOURCES variables in the same @code{Makefile.am},
+except for @code{BUILT_SOURCES}.  You can also specify the set of sources
+explicitly, through a _SOURCES variable and optionally a
+_SOURCES_EXCLUDE variable.  Both variables may include wildcards.
+
+This example lists the source files explicitly:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_SOURCES = src/maude.c src/util.c
address@hidden smallexample
+
+This example uses wildcards.  @code{**} stands for recursion across all
+subdirectories.  But note that only files known to Automake will be
+selected.  If you have other, unrelated files in the file system, they
+will not be selected, even if they match the wildcard.
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_SOURCES = src/**/*.c
+po_maude_pot_SOURCES_EXCLUDE = src/version.c
address@hidden smallexample
+
+When a C file is automatically generated by a tool, like @code{flex} or
address@hidden, that doesn't introduce translatable strings by itself,
+it is recommended to list in _SOURCES the real source file (ending in
address@hidden in the case of @code{flex}, or in @file{.y} in the case of
address@hidden), not the generated C file.
+
address@hidden A POT file's _COPYRIGHT_HOLDER
address@hidden A POT file's _COPYRIGHT_HOLDER
+
+Every POT file needs to carry a copyright notice designating the
+copyright holder, so that the translators know for whom they are doing
+the translation.  The copyright holder is declared like this:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_COPYRIGHT_HOLDER = Yoyodyne, Inc.
address@hidden smallexample
+
+The value of this variable is the copyright holder that gets inserted
+into the header of the POT file.  Set this to the copyright holder of the
+surrounding package.  (Note that the msgid strings, extracted from the
+package's sources,  belong to the copyright holder of the package.)
+Translators are expected to transfer the copyright for their translations
+to this person or entity, or to disclaim their copyright.  The empty
+string stands for the public domain; in this case the translators are
+expected to disclaim their copyright.
+
address@hidden A POT file's _MSGID_BUGS_ADDRESS
address@hidden A POT file's _MSGID_BUGS_ADDRESS
+
+It is important that translators can report problems that they find
+in the English messages to be translated.  A bug reporting address can
+be declared like this:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_MSGID_BUGS_ADDRESS = bug-maude@@yoyodyne.com
address@hidden smallexample
+
+The value of this variable is the email address or URL to which the
+translators shall report bugs in the untranslated strings:
+
address@hidden
address@hidden
+Strings which are not entire sentences, see the maintainer guidelines in
+the GNU gettext documentation,
address@hidden Strings, , Preparing Translatable Strings, gettext,
+GNU gettext tools}.
address@hidden
+Strings which use unclear terms or require additional context to be
+understood.
address@hidden
+Strings which make invalid assumptions about notation of date, time or
+money.
address@hidden
+Pluralisation problems.
address@hidden
+Incorrect English spelling.
address@hidden
+Incorrect formatting.
address@hidden itemize
+
+It can be your email address, or a mailing list address where translators
+can write to without being subscribed, or the URL of a web page through
+which the translators can contact you.
+
+You don't need to specify the bug reporting address here if you have
+already done so through the third argument of @code{AC_INIT}, see
address@hidden configure, , Initializing @code{configure}, autoconf,
+The Autoconf Manual}.
+
address@hidden Additional xgettext options
address@hidden Additional @code{xgettext} options
+
+Additional command-line options for the @code{xgettext} invocation can be
+specified through the _XGETTEXT_OPTIONS variable.  For example:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_XGETTEXT_OPTIONS = \
+  --keyword=_ --flag=_:1:pass-c-format \
+  --keyword=N_ --flag=N_:1:pass-c-format
address@hidden smallexample
+
address@hidden AM_XGETTEXT_OPTION
+Additional command-line options for the @code{xgettext} invocation can
+also be specified in @file{configure.ac} or in Autoconf macros, through
+the @code{AM_XGETTEXT_OPTION} macro.
+
address@hidden AM_XGETTEXT_OPTION (@var{option})
+
+The @code{AM_XGETTEXT_OPTION} macro registers a command-line option to be
+used in the invocations of @code{xgettext} for POT files.
+
+For example, if you have a source file that defines a function
address@hidden whose fifth argument is a format string, you can use
address@hidden
+AM_XGETTEXT_OPTION([--flag=error_at_line:5:c-format])
address@hidden smallexample
address@hidden
+to instruct @code{xgettext} to mark all translatable strings in
address@hidden invocations that occur as fifth argument to this function
+as @samp{c-format}.
+
+See @pxref{xgettext Invocation, , xgettext, gettext, GNU gettext tools}
+for the list of options that @code{xgettext} accepts.
address@hidden defmac
+
address@hidden Multiple xgettext invocations
address@hidden Multiple @code{xgettext} invocations
+
+If you need to construct a POT file from multiple @code{xgettext}
+invocations, for example because your package has source code in
+different programming languages and you need different options for
+each language, the easiest way to achieve this is through intermediate
+POT files that get combined into the final POT file:
+
address@hidden
+locale_POTS = maude.pot
+noinst_POTS = maude1.pot maude2.pot
+maude_pot_SOURCES = maude1.pot maude2.pot
+maude1_pot_SOURCES = maude.c
+maude2_pot_SOURCES = maude.lisp
address@hidden smallexample
+
address@hidden Message catalog translations
address@hidden Message catalog translations
+
+In the simplest case, you (the developer) receive translations through a
+translation project, such as the Translation Project
address@hidden://translationproject.org/}, the KDE localization project
address@hidden://i18n.kde.org/}, or the GNOME localization project
address@hidden://l10n.gnome.org/}, and include them in your release tarball.
+In this case, you should declare the set of translations in a _LINGUAS
+variable, like this:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_LINGUAS = de fr de_AT nl
address@hidden smallexample
+
+This variable should contain the list of languages for which a
+translation is present, using the notation @var{ll} or @address@hidden
+
+Alternatively, you can also have the translation downloaded from a
+translation project when the package is built.  This has the advantage
+that translations that were complete after the package was released can
+be included.  It has two drawbacks, however:
+
address@hidden
address@hidden
+No translations can be installed if the build machine does not have a
+connection to the Internet.
address@hidden
+No translations can be installed if the GNU gettext tools are not
+installed on the build machine.
address@hidden itemize
+
+To enable this mechanism, specify a _TP_URL variable or a _TP_RSYNC_URI
+variable, or both.  Example:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_TP_URL = http://translationproject.org/latest/
+po_maude_pot_TP_RSYNC_URI = translationproject.org::tp/latest/
address@hidden smallexample
+
+Currently, only the Translation Project is supported in this way.
+
address@hidden Message catalog installation
address@hidden Message catalog installation
+
+Normally, when you want translated message catalogs to be installed, you
+specify the POT file in the @code{locale_POTS} variable.  If you don't
+want them installed (for example, if they are part of a test suite only),
+you specify the POT file in the @code{noinst_POTS} variable.
+
address@hidden
+* A POT file's _CATALOGFORMAT::      Formats of compiled message catalogs
+* A POT file's _LOCALE_CATEGORIES::  Using different locale categories
address@hidden menu
+
address@hidden A POT file's _CATALOGFORMAT
address@hidden A POT file's _CATALOGFORMAT
+
+Before installation or distribution, message catalogs need to be compiled
+to ``compiled message catalogs''.  The format of these compiled message
+catalogs depends on the programming language or base runtime libraries
+that your package is using.  Several formats are supported:
+
address@hidden @code
address@hidden mo
+This is the GNU @code{.mo} format.  It is used for C, C++, and many other
+programming languages.
address@hidden qm
+This is the message catalog format of the Qt library
address@hidden://qt.nokia.com/}.
address@hidden properties
+This is the Java @code{.properties} format.  For details, see
address@hidden, , Java, gettext, GNU gettext tools}.
address@hidden class
+This is the Java @code{.class} format.  For details, see
address@hidden, , Java, gettext, GNU gettext tools}.
address@hidden resources.dll
+This is the message catalog format of C#.
address@hidden msg
+This is the message catalog format of Tcl.
address@hidden table
+
+For example:
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_CATALOGFORMAT = msg
address@hidden smallexample
+
+If you don't specify this variable, the default is @code{mo}.
+
address@hidden A POT file's _LOCALE_CATEGORIES
address@hidden A POT file's _LOCALE_CATEGORIES
+
+A locale consists of multiple functional areas called ``locale
+categories''.  The locale category for message output is called
address@hidden, the locale category for date and time formatting is
+called @code{LC_TIME}, and so on.
+(@pxref{Aspects, , Aspects in Native Language Support, gettext,
+GNU gettext tools}).  Message catalogs are usually used only with
+the @code{LC_MESSAGES} locale category, but they can rarely also
+be useful in other categories.  This set of categories can be specified
+through the _LOCALE_CATEGORIES variable.  For example, GNU coreutils uses
+its message catalogs for the @code{LC_MESSAGES} and @code{LC_TIME}
+locale categories:
+
address@hidden
+po_coreutils_pot_LOCALE_CATEGORIES = LC_MESSAGES LC_TIME
address@hidden smallexample
+
+This variable allows a coreutils program, when run in an environment
+where the @code{LC_TIME} locale category is bound to a different locale
+than the @code{LC_MESSAGE} locale category, to properly obey the
address@hidden setting.
+
+If not specified, this variable defaults to @code{LC_MESSAGES} only.
+
address@hidden Other POT files details
address@hidden Other POT files details
+
+There are two more variables associated with a POT file.
+
address@hidden
+* A POT file's _USE_MSGCTXT::        Declaring whether msgctxt is used
+* A POT file's _MSGMERGE_OPTIONS::   Specifying options for msgmerge
address@hidden menu
+
address@hidden A POT file's _USE_MSGCTXT
address@hidden A POT file's _USE_MSGCTXT
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_USE_MSGCTXT = no
address@hidden smallexample
+
+This tells whether the POT file contains messages with an @code{msgctxt}
+context.  Possible values are ``yes'' and ``no''.  Set this to yes if the
+package uses functions taking also a message context, like the
address@hidden function, or if in _XGETTEXT_OPTIONS you define keywords
+with a context argument.  If set to yes, GNU gettext tools older than
+gettext 0.15 will not be considered.  The default is @code{yes}.
+
address@hidden A POT file's _MSGMERGE_OPTIONS
address@hidden A POT file's _MSGMERGE_OPTIONS
+
address@hidden
+locale_POTS = po/maude.pot
+po_maude_pot_MSGMERGE_OPTIONS =
address@hidden smallexample
+
+These options get passed to @code{msgmerge}.
+Useful options are in particular:
+
address@hidden @code
address@hidden --previous
+to keep previous msgids of translated messages,
address@hidden --quiet
+to reduce the verbosity.
address@hidden table
+
+If this variable contains @code{--previous}, GNU gettext tools older than
+gettext 0.16 will not be considered.  The default is @code{--previous}.
+
+
 @node Other GNU Tools
 @chapter Other GNU Tools