[lmi] Cross-compiling lmi from Linux to MSW

[Vadim Zeitlin]

 Hello,

 Here is what I did to cross-compile lmi from a Debian Jessie system. First
the dependencies needed for building lmi itself (e.g. this doesn't include
cgicc). Notice that I'm unpacking all of them under ~/src/3rdparty,
building them under ~/build and installing under ~/msw/i686-w64-mingw32 but
this is, of course, completely arbitrary and you're free to use any other
directories. I do recommend not building in the source directory however,
even in spite of the fact that (spoiler alert) it resulted in a problem
during the libxslt build.


boost
-----

This was the most complicated part. The really aggravating thing is that I
could cross-compile the latest 1.60 version without too many problems: for
the future reference (and because I don't know whether this is really
documented anywhere, at least I didn't find anything else than indirect
references), let me just note that you need to do

        $ echo 'using gcc : mingw : i686-w64-mingw32-g++ ;' > ~/user-config.jam

and then you can build bjam and run

        $ ./tools/build/src/engine/bin.linuxx86_64/b2 \
            toolset=i686-w64-mingw32 target-os=windows filesystem regex

to build the libraries. However with the 1.38 version used by lmi currently
I couldn't find any way to do it and the ancient version of bjam used in it
doesn't seem to have had any documentation at all or, at least, it's
impossible to find it by now. Also, using bjam2 with boost 1.38 doesn't
work neither. So I ended up just compiling everything manually which is
hardly ideal but I hope we can also update to a later Boost version soon
which will allow to cross-compile it normally and I didn't want to waste
even more time on this than I already did. Here are the commands used:

        $ cd ~/src/3rdparty
        $ wget http://downloads.sourceforge.net/boost/boost_1_38_0.tar.bz2
        $ tar xf boost_1_38_0.tar.bz2
        $ cd boost_1_38_0
        $ for d in regex system filesystem; \
            (cd libs/$d/src && \
            i686-w64-mingw32-g++ -I../../.. -c *.cpp && \
            i686-w64-mingw32-ar rc ~/msw/i686-w64-mingw32/libboost_$d.a *.o)
        $ cp -a boost ~/msw/i686-w64-mingw32/include

Notice that the commands above do _not_ use "-std=c++11" for compiling for
the good reason that Boost 1.38 doesn't build with this option (there are
many, many errors due to the fact that this old version used pre-standard
support for things like rvalue references which worked differently from the
real C++11, so there is really no chance of fixing this). Because of this I
couldn't make things compile in C++11 mode for now, we absolutely have to
update Boost for this.


libxml2
-------

I downloaded the version used in install_libxml2_libxslt.make:

        $ cd ~/src/3rdparty
        $ wget 
ftp://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/libxml2-2.6.26.tar.bz2
        $ tar xf libxml2-2.6.26.tar.bz2

and configured it from a separate build directory:

        $ mkdir ~/build/libxml2-msw
        $ cd $_
        $ ~/src/3rdparty/libxml2-2.6.26/configure \
            --disable-static --enable-shared \
            --with-{debug,schemas} --without-{iconv,modules,python,schematron} \
            --prefix=$HOME/msw/i686-w64-mingw32 \
            --build=x86_64-unknown-linux-gnu \
            --host=i686-w64-mingw32 \
            --without-threads

The first 2 lines contain the options used for the current build of libxml2
as part of lmi, so I just took them unchanged. The prefix option is, of
course, arbitrary, but you do need to use some non-standard directory to
avoid mixing up cross-compiled and native files. The next 2 lines are
almost self-explanatory too, just notice that you need to specify the
--build value too for the ancient version of configure used by this (almost
10 year old) version of libxml2, it's not used for the more recent versions
(but then I can't even regenerate configure for 2.6.26 as it uses obsolete
features support for which has been removed since then).

The last line is the really questionable one: I had to use it to work
around a compilation error in thread-related code due to configure
erroneously detecting both pthreads and Win32 threads. As I said, fixing
this is not completely trivial because configure is a generated file and
I'd have to do some not completely trivial changes in configure.in to allow
using it with the current autoconf versions. As lmi doesn't use threads
currently, I just chose the simplest solution, but I don't like it at all,
threads are not something exotic and best to be avoided nowadays and it's
perfectly possible that lmi starts using them one of these days, especially
as they're part of C++11. IMO the right thing to do would be to update to a
newer libxml2 version in which this problem is fixed and not use this
option.

Anyhow, when configured with these options you can just do

        $ make -s install

and, after waiting for hundreds of warnings to scroll by, get the DLL and
the rest generated under ~/msw/i686-w64-mingw32.


libxslt
-------

This is globally similar to the above, but there is also a patch to apply:

        $ cd ~/src/3rdparty
        $ wget 
ftp://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.17.tar.bz2
        $ tar xf libxslt-1.1.17.tar.bz2
        $ patch -p1 < ~/src/lmi/libxslt-1.1.17-lmi.patch
        $ mkdir ~/build/libxslt-msw
        $ cd $_
        $ ~/src/3rdparty/libxslt-1.1.17/configure \
            --disable-static --enable-shared \
            --with-debug --without-{crypto,python} \
            --prefix=$HOME/msw/i686-w64-mingw32 \
            --build=x86_64-unknown-linux-gnu \
            --host=i686-w64-mingw32 \
            --with-libxml-prefix=$HOME/msw/i686-w64-mingw32
        $ make -s install

... and then run into a build error. It took me some time to understand
what was going on here, but it turns out that the tarball erroneously
includes one of the files generated by configure and, due to another bug in
the makefiles, it appears first in the header search path, breaking the
build with the lmi-specific options. So:

        $ rm ~/src/3rdparty/libxslt-1.1.17/libxslt/xsltconfig.h

However there is another error after this, due to calling mkdir() function
with 2 arguments when it only takes 1 under MSW. I was surprised that this
error didn't happen with the old MinGW build, but after investigating it a
little, it became clear why: although it's difficult to see among all the
other warnings, there are these lines when building libxslt:

---------------------------------- >8 --------------------------------------
security.c: In function `xsltCheckWrite':
security.c:351: warning: implicit declaration of function `mkdir'
---------------------------------- >8 --------------------------------------

i.e. mkdir() is not declared at all with 3.4.5, so there is no error about
passing a wrong number of parameters to it. With 4.9.2 we're getting this,
by contrast:

---------------------------------- >8 --------------------------------------
security.c: In function 'xsltCheckWrite':
security.c:351:13: error: too many arguments to function 'mkdir'
       ret = mkdir(directory, 0755);
             ^
In file included from /usr/share/mingw-w64/include/sys/stat.h:14:0,
                 from security.c:18:
/usr/share/mingw-w64/include/io.h:267:15: note: declared here
   int __cdecl mkdir (const char *) __MINGW_ATTRIB_DEPRECATED_MSVC2005;
               ^
---------------------------------- >8 --------------------------------------

 So I had to update the patch with the fix, backported from a more recent
libxslt version, and I include it as an attachment. At least after this
"make install" finally finishes successfully.


xmlwrapp
--------

We're currently using xmlwrapp 0.7.1 but there are a couple of problems
preventing it from being easily cross-compiled, in particular it requires a
couple of Boost library that we don't otherwise use. It would be possible
to build these libraries just for it, but considering that I also wanted to
fix the warnings about the use of std::auto_ptr<> in it, I decided to use
the latest git version instead. While I understand that you want to avoid
changing more than one thing at once, in this particular case I think the
change is very safe as there are no changes in the sources whatsoever
between 0.7.1 and the latest master of xmlwrapp, only a couple of additions
which don't affect lmi use of it. I could, of course, create a patch
against 0.7.1 but it's relatively big due to changes in the other, non
source, files, and cherry picking just the changes that we need would mean
that we'd end up with some weird intermediate version of xmlwrapp which is
neither 0.7.1 nor 0.7.2 (or 0.8.0 if this is what it becomes) which doesn't
seem ideal.

So finally I did just:

        $ git clone https://github.com/vadz/xmlwrapp.git
        $ cd xmlwrapp
        $ ./bootstrap
        $ mkdir ~/build/xmlwrapp-msw
        $ cd $_
        $ ~/src/xmlwrapp/configure \
                --disable-shared --disable-tests --disable-docs \
                --with-boost=$HOME/msw/i686-w64-mingw32 \
                --prefix=$HOME/msw/i686-w64-mingw32 \
                --host=i686-w64-mingw32 \
                PKG_CONFIG_LIBDIR=$HOME/msw/i686-w64-mingw32/lib/pkgconfig
        $ make install


wxWidgets
---------

This one was the simplest. I used the latest git sources (which should
become 3.1.0 relatively soon), but I'm quite confident that it also works
with the git commit 0b821adf903872b6d8b56630d2191c5b9c3362e5 from
install_wx.make or any other revision in between it and now.

        $ git clone https://github.com/wxWidgets/wxWidgets.git
        $ mkdir ~/build/wx-msw
        $ cd $_
        $ ~/src/wxWidgets/configure \
                --prefix=$HOME/msw/i686-w64-mingw32 --host=i686-w64-mingw32 \
                --disable-precomp-headers \
                --disable-apple_ieee --disable-compat24 --disable-fswatcher \
                --disable-gif --disable-aui --disable-mediactrl \
                --disable-propgrid --disable-ribbon --disable-richtext \
                --disable-stc --disable-webview --disable-threads \
                --enable-monolithic --without-opengl --enable-shared \
                --enable-std_iostreams --enable-stl --without-libjpeg \
                --without-libtiff --without-subdirs
        $ make install


wxPdfDoc
--------

This one was also pretty straightforward:

        $ git clone https://github.com/vadz/wxpdfdoc.git
        $ mkdir ~/build/wxpdfdoc-msw
        $ cd $_
        $ ~/src/wxpdfdoc/configure \
                --prefix=$HOME/msw/i686-w64-mingw32 --host=i686-w64-mingw32 \
                --with-wx-config=$HOME/msw/i686-w64-mingw32/bin/wx-config
        $ make install

No problems here.


lmi itself (finally!)
---------------------

After all the struggles with the dependencies, this one was pretty simple.
However some changes were still required: first, I'd like to commit a patch
removing the code unconditionally appending "mgw" suffix to the Boost
libraries when building using MinGW compiler from configure, it's always
possible to specify it explicitly using --with-boost-toolkit option, but
it's impossible to specify it as being empty using it. Second, we need to
add the "force linking" stuff to alert_wx.cpp and main_wx.cpp to ensure
that the alert function pointers are set correctly in lmi_wx.exe as
otherwise they were not, resulting in an error message and abort, for the
same reason as in the MSVS build: alert_wx object file was simply omitted
entirely by the linker. Both patches are attached.

Building lmi is pretty simple and required just disabling some of the
warnings from Boost headers that were turned into errors by -Werror
otherwise (and were very annoying even without -Werror anyhow):

        $ mkdir ~/build/lmi-msw
        $ cd $_
        $  PATH=$HOME/msw/i686-w64-mingw32/bin:$PATH ~/src/lmi/configure \
                --prefix=$HOME/msw/i686-w64-mingw32 --host=i686-w64-mingw32 \
                CPPFLAGS=-I$HOME/msw/i686-w64-mingw32/include \
                LDFLAGS=-L$HOME/msw/i686-w64-mingw32/lib \
                CXXFLAGS='-Wno-unused-local-typedefs -Wno-unused-variable'
        $ make install

The only remaining problem is that there are a few extra steps to be done
before the program can be actually used:

- Copy (or move, to save space) the wx DLLs from lib to bin so that they're
  found without having to explicitly set the PATH for them under MSW:

        $ mv ~/msw/i686-w64-mingw32/lib/*.dll ~/msw/i686-w64-mingw32/bin

- Copy the required MinGW to the same directory:

        $ cp 
/usr/lib/gcc/i686-w64-mingw32/4.9-win32/lib{stdc++-6.dll,gcc_s_sjlj-1}.dll 
~/msw/i686-w64-mingw32/bin

- Generate the product files manually (this needs to be done under MSW,
  either after copying the files or by sharing them with the VM):

        > cd ~/msw/i686-w64-mingw32/share/lmi
        > ../../bin/product_files.exe

- Create configurable_settings.xml file in ~/msw/i686-w64-mingw32/bin.

But after doing all this I could run it just fine from both inside a VM and
from my real/physical Windows box (after rsync-ing the files to it, of
course).


 To summarize, I can now cross-compile lmi easily, the main remaining
problem is that I still can't do it in C++11 mode because our version of
Boost is too old to support being built with a C++11 compiler. I'd like to
change this a.s.a.p., I already have the minimal Boost.Filesystem patches
to make the code compile with the latest Boost. And ideally we'd also
replace parts of Boost which are now part of C++11 standard library itself
(e.g. shared_ptr, regex, ...) with the corresponding standard classes. But
this, of course, would require more changes.

 Please let me know how would you like to proceed, but in the meanwhile
I'll stop here (and it's even still January 13 in your time zone so I
almost fulfilled my promise...)

 Regards,
VZ

0001-Don-t-look-for-Boost-libraries-with-mgw-prefix-only-.patch
Description: Text document

0002-Force-linking-wxWidgets-alert-functions-into-the-mai.patch
Description: Text document