Re: Are frozen files really that hot?

[Eric Blake]

On 07/01/2014 03:25 PM, Daniel Goldman wrote:
> Hi,
> 
> The manual says: "Later calls, containing the -R option, are able to
> reload the internal state of m4, from base.m4f, prior to reading any
> other input files. This means instead of starting with a virgin copy of
> m4, input will be read after having effectively recovered the effect of
> a prior run. In our example, the effect is the same as if file base.m4
> has been read anew. However, this effect is achieved a lot faster."
> 
> "A lot faster" sounds promising. But also kind of vague. 100% faster?
> 200% faster? And of course the frozen file just affects the loading
> phase. If the loading phase is just a fraction of the processing cycle,
> it might not matter much.

It depends on how complex those macros were.

> 
> That's a 31% improvement. (9.42 - 6.54) / 9.42 = .306
> 
> Or one could say the non-frozen test takes 44% longer.

Yay that you were able to measure a difference for your test case.

> 
> That's better than nothing, but I was hoping for better. I realize this
> is just one test. And sorry not feasible for me to provide the details,
> so a synthetic benchmark would also be helpful. Are there other
> real-world (or synthetic) benchmarks for frozen files, or any cases
> where frozen files are really that helpful for some task?

Yes.  At one point, autoconf had some O(n^2) (and even O(n^3)) macro
definitions, some of which were encountered in the raw file and thus
caused some noticeable delays before even getting to the user's
configure.ac file.  The reason that GNU m4 introduced frozen files was
because autoconf had SO MANY MACRO definitions, and that some of those
definitions were computed definitions created by quadratic algorithms,
where loading the frozen file really did make a difference (it's much
faster to read in a macro's definition in frozen form than it is to
parse one word at a time to recompute the same definition).  This was
before GNU m4 1.4 was released.  Since then, two things have improved -
m4's parsing speed has been improved (for example, it now uses
getc_unlocked instead of getc, which itself had a noticeable speedup on
processing raw files), and autoconf has moved away from O(n^2)
algorithms (while it still defines a LOT of macros that still get loaded
faster from frozen, it is no longer as complex to compute what those
macros will contain).  So I'm not sure if you will see better or worse
numbers from autoconf, but I do know that autoconf still makes heavy use
of loading frozen files so that it gets to parsing the user's
configure.ac sooner.

> 
> If only minimal effort is required related to frozen files (hard for me
> to believe), then I guess there is no problem. Otherwise, I am wondering
> if frozen files are worth it. At this point, based on my benchmarks, I
> don't plan to use frozen files.

They're not mandatory to use.  But at this point, we can't rip it out of
m4 - there are users that depend on it.  The code is designed to not
penalize people that aren't using it.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

signature.asc
Description: OpenPGP digital signature