Re: moving po/Makefile.in.in to automake

[Alexandre Duret-Lutz]

>>> "Bruno" == Bruno Haible <address@hidden> writes:

 Bruno> Hi Alexandre et al.,

Hi Bruno, sorry for the delay.

[...]

 Bruno> Now automake seems mature enough and widely enough
 Bruno> used. I'd like to ask for your cooperation in moving the
 Bruno> functionality of po/Makefile.in.in to automake.

Do you really mean to _move_ the functionality, in the sense
that po/Makefile.in.in will no longer exist and people will have
to use Automake or write their own rules?  Or would you simply
like that Automake provides similar functionality, and still
support po/Makefile.in.in?

It does make some difference.  In the latter case, we do not
have to cover all the subtleties of Makefile.in.in.  Supporting
the usual setups would already be very useful.

 Bruno> This mail is a first step in doing this, but I need your
 Bruno> cooperation because I'm not inclined to learn Perl.

This I can understand :)

[...]

 Bruno> As far as I've understood, the philosophy of automake
 Bruno> (versus make) is <<declarative specification of
 Bruno> frequently used tasks>> (versus <<rule-based
 Bruno> specification of any tasks>>).

 Bruno> Right? So we need to define a set of Makefile.am
 Bruno> variables that will allow the developer to specify the
 Bruno> parametrization of the PO directory. More about this
 Bruno> below.

It might be nice to devise a `dir_PRIMARY' syntax, for
uniformity with the other installable kinds of files.  This way
maybe we can even get rid of "the PO directory" assumption (can
we?).

I'm thinking of something along the lines of

  locale_POTS = foo.pot
  foo_pot_SOURCES = a.c a.h b.c b.h ...
  foo_pot_LANGFILE = LINGUAS
  foo_pot_XGETTEXTFLAGS = --keyword=_ --keyword=N_
  etc.

(For a moment I though about using `locale_DOMAINS = foo', but I
realized that foo_SOURCES also lists the source of the foo binary.)

It's not clear what happens to the po/gmo files when multiple
domains are used in the same directory.  Maybe we can rename
them.  Maybe this should be disallowed (but it would still
make sense to write `locale_POTS = foo/foo.pot bar/bar.pot').

 Bruno> Then a question about practical procedures: Since the
 Bruno> .m4 files that provide the prerequisites for automake
 Bruno> generated rules should evolve in sync with automake, and
 Bruno> since we want to avoid coupling automake releases and
 Bruno> gettext releases (like "automake 2.0 needs gettext
 Bruno> 0.14.1 to 0.14.3 but no older and no newer version"), it
 Bruno> does probably mean that some of these 25 .m4 files get
 Bruno> moved from gettext to automake. Or is there another
 Bruno> mechanism that would allow me to continue maintaining
 Bruno> these .m4 files while not being an automake developer in
 Bruno> the general sense?

We could do as we do with libtool, it does not seem to be really
different.  When Automake outputs rules that use $(LIBTOOL), it
just requires the LIBTOOL variable to be defined.
lib/Automake/Variable.pm has a list of known variable/macro
pairs so Automake will suggest using AC_PROG_LIBTOOL, but users
are free to define LIBTOOL differently.

For gettext rules we could do similarly: use $(MSGFMT),
$(MSGMERGE), etc., in rules, and require these variables to be
defined or suggest calling AM_GNU_GETTEXT if they are not.

We need to work out the list of all these variables and ensure
future version of gettext always define them, but that does not
seem burdensome.

 Bruno> Would it be feasible for gettext to install some .m4
 Bruno> files into $prefix/share/aclocal/ and some .am files
 Bruno> into $prefix/share/automake/am/ ? [Similarly, would it
 Bruno> be feasible for the libtool people to maintain the
 Bruno> libtool specific files in $prefix/share/automake/am/
 Bruno> etc.? Such conventions could greatly extend the
 Bruno> flexibility of automake.]

The problem is that an *.am file means nothing alone, half of
the logic is written in Perl in automake itself (which macros to
require, when to read the *.am files, how to define some special
variables, etc.).

There seem to be two complementary directions to improve this.

  1. move more of the logic into *.am

     Akim worked a lot on this between Automake 1.4 and 1.5.
     Probably things more deserve a syntax in `*.am' files.
     However the automake parsers (there are two of them -- they
     have yet to be unified) are regex-based, and it would be
     difficult to support elaborate constructs such as for-loops
     around rules...  Akim likes to suggest someone should
     develop a Perl skeleton for Bison; that way we could have
     a real parser in Automake :)

  2. modularize the Perl code itself.

     The ultimate goal is to support each _PRIMARY in its own
     module (and then think about third-party plug-ins).
     Unfortunately we must do this from the bottom: modularize
     anything that a plug-in might use.  Some work has been done
     toward this in 1.7 and 1.8, but this is hard and the road
     seems very very long.  (Actually I see this as a direction
     to follow to clean the code, not as a goal to reach.)

As far as gettext is concerned, we should not count on anything
like this.

One way to further decouple the Gettext tools from the Makefile
rules is to supply tools that achieve the most complex
operations.  This would also help people who write their own
Makefile.in.  (Think about install-info or libtool
--mode=install.)  I have not looked at the po/Makefile in
details, but at a first glance I don't see anything that would
require this.  Except the $(DOMAIN).pot-update rule maybe: it
seems it would be easier if xgettext did not overwrite its
output needlessly, that would save the shell code and the sed
macro (that's also how `aclocal' and `automake --no-force'
behave).

 Bruno> Going into some details about po/Makefile.am:
 Bruno> - It has some variable definitions, which should remain
 Bruno> in this place.
 Bruno> - It has some definitions of deduced variables, which
 Bruno> are easy to define using GNU make syntax, but require
 Bruno> some postprocessing of the Makefile.in generated by
 Bruno> automake. This is quite ugly. Could automake be extended
 Bruno> to allow a subset of the GNU make $(...) primitives and
 Bruno> implement them by code inside config.status?

This might be useful in other places too but I don't know how
this can be done generally.  We'd have to emulate variables
expansion on currently supported features (notably nested
variables and $(foo:bar=baz) substitutions).  This looks hard.

On the other hand, it seems some of these derived variables can
be computed at make-time.

Adding a suffix can be done with

UPDATEPOFILES = $(LINGUAS:=.po-update)
DUMMYPOFILES = $(LINGUAS:=.nop)

I don't know any portably way to add a prefix, but for
$(srcdir)/ we don't have to.  Dependencies which are not
explicit targets do not need a $(srcdir)/ prefix, VPATH
will take care of them.  So we can prefix the filenames with
$(srcdir)/ inside the rules where they are used.  This seems
to apply to POFILES and GMOFILES.

As far as POTFILES_DEPS is concerned, I'm tempted to say that we
should get rid of this variable, and let users specify relative
paths in
   foo_pot_SOURCES
(or whatever we call this variable)

 Bruno> - It has some rule definitions, which are typical business for
 Bruno> automake's .am files.

 Bruno> The po/Makefile.am presented here works for C and
 Bruno> C++. For other programming languages and environments
 Bruno> some adjustments must be made:

 Bruno> - For different environments, the default
 Bruno> XGETTEXT_OPTIONS are different.

 Bruno> - For Qt programs, GMOFILES is replaced with QMFILES,
 Bruno> the .po.gmo rule is replaced with a .po.qm rule, the
 Bruno> installation rules and DISTCLEANFILES and EXTRA_DIST are
 Bruno> different.

Does the name of the variable matters?  Wouldn't it be enough to
have a @GMOEXT@ variable that is set to .gmo or .qm?

(Incidentally, $(LINGUAS:address@hidden@) is a case where we can't use 
$(GMOEXT).)

 Bruno> - For Java programs, GMOFILES is replaced with
 Bruno> PROPERTIESFILES or CLASSFILES (depending on the
 Bruno> preferences of the package), the .po.gmo rule is
 Bruno> removed, all installation rules are removed, and the
 Bruno> target update-gmo is replaced with a target
 Bruno> 'update-properties' or 'update-classes'.

 Bruno> - For Tcl programs, GMOFILES is replaced with MSGFILES,
 Bruno> the .po.gmo rule is removed, the installation rules and
 Bruno> DISTCLEANFILES, MAINTAINERCLEANFILES, EXTRA_DIST are
 Bruno> different, and the target update-gmo is replaced with
 Bruno> update-msg.

So much for uniformity :(

 Bruno> Now here is my first proposal for a set of Makefile.am variables needed
 Bruno> to customize a PO directory. Please comment on it!

 Bruno> - POTFILES: List of files which contain translatable
 Bruno> strings, relative to $(top_srcdir); mandatory. (Actually
 Bruno> 'POTFILES' is a misnomer; it should better be called
 Bruno> 'I18N_SOURCES' or something like that.)
 Bruno> - DOMAIN: The gettext domain; mandatory.
 Bruno> - XGETTEXT_OPTIONS: flags passed to xgettext (FIXME: How to accomodate
 Bruno> directories which need several xgettext invocations?); mandatory.

Shall we support a _MSGADD variable comparable to _LIBADD and _LDADD?

  nodist_noinst_POTS = bar.pot baz.pot
  bar_pot_SOURCES = x.py y.py
  bar_pot_XGETTEXTFLAGS = --language=Python
  baz_pot_SOURCES = x.el y.el
  baz_pot_XGETTEXTFLAGS = --language=EmacsLisp

  locale_POTS = foo.pot
  foo_pot_SOURCES = a.c a.h b.c b.h ...
  foo_pot_LANGFILE = LINGUAS
  foo_pot_XGETTEXTFLAGS = --keyword=_ --keyword=N_
  foo_pot_MSGADD = bar.pot baz.pot


(I tend to think it would be clearer if the syntax we choose can
distinguish *.pot file generation from *.gmo/*.qm compilation
and installation; but I don't know how to do this.)

 Bruno> - COPYRIGHT_HOLDER: gets inserted in the header of the
 Bruno> POT file; mandatory.
 Bruno> - MSGID_BUGS_ADDRESS: maintainer email address; should default to
 Bruno> AC_PACKAGE_BUGREPORT.
 Bruno> - EXTRA_LOCALE_CATEGORIES: a weirdness designed for coreutils; should
 Bruno> default to empty.

 Bruno> - AUTOMAKE_OPTIONS: one of po-qt, po-java-properties,
 Bruno> po-java-class, po-tcl - influences the kind of rules
 Bruno> generated in the po/Makefile.in.

Apart from QT which looks like a naming issue, the other three
really look like they deserve their own rules.  It's not clear
to me how worth it is to merge them with the usual rules.  Maybe
we can invent a new _PRIMARY for them later?

[...]
 Bruno> MSGMERGE = msgmerge
 Bruno> MSGMERGE_UPDATE = @MSGMERGE@ --update
[...]

This just caught my eyes.  Is it meant?  Note that when this
code is moved into an Automake file, the `MSGMERGE = msgmerge'
assignment will be overridden by `MSGMERGE = @MSGMERGE' because
AC_SUBSTed variables always override Automake defined variables.
(When it is in a Makefile.am the converse happens: Makefile.am
definitions override AC_SUBSTed variables.)
-- 
Alexandre Duret-Lutz