[lmi] Compiling takes longer with gcc-4.9.2

[Greg Chicares]

My local tree contains makefile changes to use MinGW-w64 gcc-4.9.2,
with '-std=c++11', some new warning flags, and various other
adjustments. First I compile wx and wxPdfDoc:

make $coefficiency -f install_wx.make >../log 2>&1
  out of memory

This is 32-bit msw-xp in a VM, $coefficiency being '--jobs=16', so
I reduce that and it succeeds:

make --jobs=8 -f install_wx.make > ../log 2>&1
2662.23s user 1594.59s system 475% cpu 14:55.38 total

make --jobs=8 -f install_wxpdfdoc.make > ../log 2>&1
446.34s user 182.72s system 393% cpu 2:39.91 total

Now I do a complete rebuild of lmi, which I very recently measured
with mingw.org's native gcc-3.4.5 as follows:

  --jobs=16
15.89s user 26.48s system 21% cpu 3:18.89 total
  --jobs=1
16.13s user 19.98s system 3% cpu 17:55.20 total

...and here are results for MinGW-w64 gcc-4.9.2 as installed by
lmi's current 'install_cygwin.bat' (package 'mingw64-i686-gcc-g++'):

make --jobs=2 install check_physical_closure > ../log 2>&1
1300.82s user 544.17s system 193% cpu 15:54.30 total
make --jobs=3 install check_physical_closure > ../log 2>&1
1306.38s user 559.29s system 269% cpu 11:32.61 total
make --jobs=4 install check_physical_closure > ../log 2>&1
1324.14s user 581.83s system 341% cpu 9:17.47 total
make --jobs=5 install check_physical_closure > ../log 2>&1
1340.61s user 637.67s system 349% cpu 9:26.66 total
make --jobs=6 install check_physical_closure > ../log 2>&1
1378.85s user 739.61s system 303% cpu 11:38.62 total
make --jobs=8 install check_physical_closure > ../log 2>&1
1465.71s user 969.62s system 248% cpu 16:20.89 total
make --jobs=16 install check_physical_closure > ../log 2>&1
1424.92s user 969.01s system 200% cpu 19:52.47 total

It takes three times as long, which is not an argument in favor of
upgrading. Can we improve the build time?

I can't figure out why the best result comes from '--jobs=4'. If the
number of CPUs isn't the bottleneck, what is? disk? RAM? CPU cache?
Here are some experiments...

If gcc-4.9.2 needs more RAM, this might help...

  --param ggc-min-expand=25 --param ggc-min-heapsize=32768
make --jobs=4 install check_physical_closure > ../log 2>&1
1474.91s user 565.68s system 351% cpu 9:41.12 total

...but it doesn't. Perhaps gcc-4.9.2 writes more or larger
intermediate files, it which case this might help...

  -pipe
make --jobs=4 install check_physical_closure > ../log 2>&1
1311.62s user 584.71s system 343% cpu 9:11.69 total
make --jobs=6 install check_physical_closure > ../log 2>&1
1377.05s user 731.70s system 300% cpu 11:40.95 total

...but it hardly makes a difference (and I think '-pipe' is not
widely used, so I worry that it might have subtle defects that
go unnoticed).

Maybe it's just silly to allocate 16 virtual CPUs to a 32-bit msw-xp
guest; let's try eight:

  VM: 8 CPUs
make --jobs=4 install check_physical_closure > ../log 2>&1
1320.64s user 571.95s system 332% cpu 9:28.67 total

That's no help. Maybe it's NUMA, so I tried letting 'libvirt' choose
the pinning (which would be meaningless with 16 vCPUs):

  NUMA pinning 4-7,12-15
make --jobs=4 install check_physical_closure > ../log 2>&1
1398.38s user 608.19s system 347% cpu 9:37.75 total

...but that didn't help.

This article:
  http://stackoverflow.com/questions/21029281/why-is-gcc-4-9-trunk-so-slow
suggests testing whether this compiler was build with extra diagnostics;
it wasn't.

This one seems relevant:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60976
| Compilation with G++ 4.9.0 is 2-3 times slower than with 4.8.2
It sounds like changes they made to conform to C++11 ruined the speed,
so let's try '-std=c++03':

make --jobs=8 -f install_wx.make > ../log 2>&1
2108.11s user 1488.69s system 446% cpu 13:26.20 total
make --jobs=8 -f install_wxpdfdoc.make > ../log 2>&1
347.15s user 156.70s system 368% cpu 2:16.64 total

For lmi, I experimentally pass
  treat_warnings_as_errors='-Werror'
to remove '-pedantic-errors' for the nonce as a regretful workaround for a
single wxPdfDoc variadic macro (which I should hope can be worked around
better by patching wxPdfDoc), and...

make --jobs=8 treat_warnings_as_errors='-Werror' install  > ../log 2>&1
1118.15s user 801.12s system 221% cpu 14:25.30 total
make --jobs=4 treat_warnings_as_errors='-Werror' install  > ../log 2>&1
1015.40s user 479.53s system 330% cpu 7:31.88 total

The bottom line:
  3:19 = 199s gcc-3.4.5, std=gnu++98
  7:32 = 452s gcc-4.9.2, std=c++03
  9:18 = 558s gcc-4.9.2, std=c++11

Vadim, can you make any sense of my timing data above, or think of any
way to make a newer version of gcc compile lmi faster?

This (4.9.2) is the latest MinGW-w64 package that Cygwin offers (they
have 4.9.3 and 5.2.0 versions that target Cygwin, not MinGW). Here:
  http://sourceforge.net/projects/mingw-w64/
a 4.9.4-20151215 version is offered; would that native build be better
for us? (I'm not really sure how their Cygwin package differs.)