automake-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallel compression in 'make dist'


From: Peter Breitenlohner
Subject: Re: parallel compression in 'make dist'
Date: Mon, 12 Oct 2009 10:32:07 +0200 (CEST)
User-agent: Alpine 2.00 (LNX 1167 2008-08-23)

On Mon, 12 Oct 2009, Ralf Wildenhues wrote:

* Peter Breitenlohner wrote on Wed, Sep 16, 2009 at 10:11:58AM CEST:
Another completely unrelated item:

it would be nice if Automake could run gzip and bzip2 (and others) in
parallel when creating the distribution tarballs.  In a parallel make that
could save quite some time when running dist(check) for a largish project.

Feasible?

I think so.  Can you try this patch and check whether it improves `make
dist' time for you significantly?  Thanks.

Hi Ralf,

this looks very nice. Here my timing results for 'make -j dist' in the TeX
Live tree:
          A      B      C
  real:  2:59   2:21   2:23
  user:  2:43   2:40   3:27
  sys:   0:08   0:06   0:06
where A creates .gz and .xz with Automake-1.11, B the same with your patch
"applied" manually to Makefile{.in,}, and C the same for .gz, .bz2, and .xz. The timing for the individual compressors is 0:28 for gzip (exactly the
difference in real time between A and B), 0:44 for bizp2 (about the
difference in user time between B and C), and 2:09 for xz.

All tests on a (more or less idle) Intel Core2 Quad with i686-linux-gnu.

I did some quick .....

indicating that the extra make recursion seems to be acceptable, and the
optimization worthwhile for some packages....

I fully agree. For small and flat packages everything is fast, and for large
and/or deep packages this extra recursion ought to be negligible.

... Notably, -j2 isn't helpful
for GM, as it starts the slowest encoder late only.

I could think of one more improvement for -j2: running the slowest
encoder first (provided that can be achieved easily).

BTW, this is definitely a 1.12-only change:

Of course.

Regards
Peter Breitenlohner <address@hidden>




reply via email to

[Prev in Thread] Current Thread [Next in Thread]