bug#20082: new warning from ar on rawhide systems

[Pavel Raiskup]

On Wednesday 11 of March 2015 10:07:33 Eric Blake wrote:
> When compiling libvirt (and probably other packages) on Fedora rawhide
> systems, I'm seeing a new warning message on every link line.
>
> # rpm -q libtool gcc binutils
> libtool-2.4.6-1.fc23.x86_64
> gcc-5.0.0-0.16.fc23.x86_64
> binutils-2.25-6.fc23.x86_64
>
> For an example of the warning during a 'make V=1':
>
> > libtool: link: rm -fr  .libs/libvirt_driver_qemu_impl.a 
> > .libs/libvirt_driver_qemu_impl.la
> > libtool: link: ar cru .libs/libvirt_driver_qemu_impl.a 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_agent.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_capabilities.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_command.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_domain.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_cgroup.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_hostdev.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_hotplug.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_conf.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_process.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_migration.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_monitor.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_monitor_text.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_monitor_json.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_driver.o 
> > qemu/.libs/libvirt_driver_qemu_impl_la-qemu_interface.o
> > ar: `u' modifier ignored since `D' is the default (see `U')
> > libtool: link: ranlib .libs/libvirt_driver_qemu_impl.a
>
> Reading 'man ar', it looks like a relatively new binutils invention, one
> that Fedora chose to flip on by default, but also one that might be
> worth using even when ar is built with 'U' rather than 'D' as the default:
>
> >        D   Operate in deterministic mode.  When adding files and the archive
> >            index use zero for UIDs, GIDs, timestamps, and use consistent 
> > file
> >            modes for all files.  When this option is used, if ar is used 
> > with
> >            identical options and identical input files, multiple runs will
> >            create identical output files regardless of the input files'
> >            owners, groups, file modes, or modification times.
> >
> >            If binutils was configured with --enable-deterministic-archives,
> >            then this mode is on by default.  It can be disabled with the U
> >            modifier, below.
>
> Is it worth teaching automake to change ARFLAGS to be 'crD' instead of
> 'cru' when it is detected that ar is new enough to support deterministic
> libraries, in part to shut up the warning message being printed on every
> single libtool link line?  (Note that I would explicitly include 'D',
> because even though Fedora chose to make 'D' the default, other distros
> may choose to make 'U' the default).  Or conversely, do we want ARFLAGS
> to be 'cruU', to force non-deterministic mode, since any speedups made
> possible by timestamp comparison ('u') are only possible in
> non-deterministic mode?  Does the use of 'u' buy us much measurable time
> when repeatedly and incrementally linking large libraries, where the new
> 'D' mode is forced to link everything instead of just the new inputs?

>From GNU ar code I assume that having the 'u' might save some file opening
and closing, because only files with newer timestamp are opened to be
copyied into new archive with 'u'.  Files with mtime lower than the
corresponding archive member are copyied directly from (already opened)
old archive.

If at least one member was updated on the disk before calling 'ar cru',
_new_ archive is created (no in place update of archive).  When no file
was changed, GNU ar does not touch existing archive.  In Makefile
(precisely defined dependencies) this is not useful optimization however..

So probably the worst slowdown would be an 'ar' task to update archive
consisting of a big amount of small files when only one has changed.

FTR, Automake uses 'cru' since at least ~1994 initial commit in git, cmake
uses 'cr' only.

> And of course there's the question of how easy is it to detect that ar
> is new enough to understand the 'D'/'U' dichotomy?

Not sure.  Probably ./configure check like '--version works' && 'GNU in
--version' && 'ar crD' could be enough..

However, having best-effort determinism in Automake does not guarantee
determinism of final result..  Is it worth the trouble?  I could imagine
some general './configure --enable-deterministic-build', but then I would
expect the configure to fail if the 'D' option is not available..

Having used 'cru' with optional 'cruU' could have at least a small benefit
that the default behavior would behave consistently at least on boxes
running GNU ar..  which is not the actual state.

IMO, removing the 'u' optimization for everybody is the easiest thing to
do.  Its suboptimal because 'u' is probably portable enough it has worked
for automake for 20+ year.. and we would remove it just to silence warning
on some platforms .. but ..

> I've already asked on libtool, and they suggested it is an automake issue.

The link to make it complete:
http://lists.gnu.org/archive/html/bug-libtool/2015-02/msg00014.html

Pavel