[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
parallelization of ./configure compiler test processes
From: |
Danny McClanahan |
Subject: |
parallelization of ./configure compiler test processes |
Date: |
Wed, 29 Mar 2023 20:12:49 +0000 |
Hello autoconf,
I work on a cross-platform package manager named spack (https://spack.io) which
builds lots of gnu software from source and has fantastic support for autotools
projects. Because spack provides a shell script `cc` to wrap the compiler, each
invocation of `cc` for feature tests executed by `./configure` takes a little
bit longer than normal, so configuring projects that necessarily have a lot of
feature tests takes much longer in spack (particularly `gettext`, which we use
as a benchmark in this change: https://github.com/spack/spack/pull/26259).
However, we can fix that additional overhead ourselves without any changes in
autoconf, by generating our `cc` wrapper instead of doing any logic in the
shell script. The reason I messaged this board is because of a separate idea
that the above situation made me start thinking about: *parallelizing feature
test executions*, in order to speed up `./configure`.
So a few questions:
1. Are there any intrinsic blockers to parallelizing the generated feature
tests that execute in an autotools `./configure` script?
- For example, I've been told that feature tests currently write to a
single output file, which would get clobbered if we were to naively parallelize
the test execution, but I was hoping that each test could be made to write to a
temp file instead if that's true.
2. Which codebase (autoconf, automake, m4, ?) does the work of generating the
script that executes tests in serial, and where in that codebase does this
occur?
- I've been perusing clones of the autoconf and automake codebases and I've
been unable to locate the logic that actually executes each test in sequence.
3. How should we expose the option to execute tests in parallel?
- In order to serve the purpose of improving `./configure` invocation
performance, we would probably want to avoid requiring an `autoreconf` (spack
avoids executing `autoreconf` wherever possible).
- Possibly an option `autoreconf
--experimental-also-generate-parallel-tests`, which would enable the end user
to execute `./configure --experimental-execute-parallel-tests`?
Please feel free to link me to any existing content/discussions on this if I've
missed them, or redirect me to another mailing list. I'm usually pretty good at
figuring things out on my own but have been having some difficulty getting
started here.
Thanks so much,
Danny
- parallelization of ./configure compiler test processes,
Danny McClanahan <=