Re: libtool and mkoctfile

[Jaroslav Hajek]

On Thu, Nov 5, 2009 at 11:48 PM, Thomas Weber
<address@hidden> wrote:
> Hi,
>
> I'm picking Jaroslav's message for answering, but I hope I've added
> everybody currently involved into the CC.
>
> On Wed, Nov 04, 2009 at 10:52:56AM +0100, Jaroslav Hajek wrote:
>> On Wed, Nov 4, 2009 at 9:50 AM, Benjamin Lindner <address@hidden> wrote:
>> >
>> >> I would like to make the mkoctfile script use libtool to create .oct
>> >> files, the same as we will soon be using in the Makefiles that build
>> >> the .oct files that are part of Octave.  I think I have most of the
>> >> details worked out, but one problem remains.  Libtool is a Unix shell
>> >> script, so we will need to do something for Windows systems.  Is it
>> >> reasonable for Octave to require a Unix command shell and the
>> >> associated commands that libtool requires for building .oct files?
>> >
>> > Frankly, no. I'd like to return the question by asking is it *really* 
>> > necessary to rely on a unix-sh-interpreter?
>> > I see mkoctfile as an integral part of Octave an if mkoctfile requires a 
>> > unix-layer to run, then, well, you won't have a native windows binary any 
>> > more.
>> > And I wouldn't see the point in providing mingw binaries with the 
>> > additional necessity of a unix shell, when there's cygwin available which 
>> > does exactly all this, and I guess much better.
>> > And what about msvc? There are no 'official' binaries, but octave can 
>> > still be built with ms compilers I guess and run as native binary.
>> >
>> > What about the future development, what about e.g. x64 binaries?
>> > I'm not sure about unix-shells-for-win64-which-are-not-cygwin. Possibly 
>> > cygwin is the better choice as unix layer - and then I'll end up again at 
>> > the question, why provide mingw32/64 along with a unix shell interpreter 
>> > when there is cygwin?
>> >
>> > Aside from the question of unix-sh-interpreter yes or no, I disagree to 
>> > using libtool, at least for windows platform. Simply because libtool does 
>> > not work properly for windows.
>> > There are problems creating shared libs, there are problems with compiler 
>> > flags, there are problems with libtool dropping libraries from the command 
>> > line at own gusto, there are problems with shared libstdc++.
>> > So I would most probably end up packaging a patched (i.e. hacked) version 
>> > of libtool which I honestly could not support because i simply don't 
>> > understand it myself.
>> > Using libtool for building octave is using it in a controlled, and kind of 
>> > encapsuled environment where one knows the odds and ends and where you 
>> > have a bunch of workaround tricks in your pocket.
>> > But distributing it and depending on it in open space, I believe will end 
>> > up with many reports of "mkoctfile does not work for me" and us saying, 
>> > "yes, we know, this is libtool".
>> >
>> >> If not, then I suppose that when Octave is built, we could try to
>> >> extract from libtool the commands that it would run, and embed those
>> >> in the mkoctfile script/program, though I'm not sure how to do that in
>> >> a reliable way.  If we do go this route, then I would like to consider
>> >> again eliminating the mkoctfile script and keeping only the C++
>> >> program.  Having both seems like asking for trouble as the two are
>> >> likely to get out of sync.
>> >>
>> >> It seems easier to me to just package a Unix shelland the necessary
>> >> commands for Windows systems, then eliminate the C++ version of
>> >> mkoctfile.
>> >
>> > I'd like to stick to the C++ version of mkoctfile and I would have no 
>> > problem with having a (windows) C++ version alongside the (common) shell 
>> > version. You are right about having two versions and them possibly 
>> > diverging, but how much development in the future do you honstly see in 
>> > mkoctfile? How much new features do you plan to add to mkoctfile? I think 
>> > that the additional effort of keeping the C++ version in pace with the 
>> > shell version small compared to forcing a shell/libtool version for 
>> > windows and maintaining it.
>> > I'd rather be one step behind with the C++ version than requiring a unix 
>> > shell for it.
>> >
>> > To me, the C++ interface is octave's biggest bonus aside of it being free 
>> > software. And to be able to use the C++ interface natively with a free 
>> > compiler available on windows, I wouldn't want to miss that.
>> >
>> >> If you object to including a Unix shell, why?
>> >
>> > I'd object to having a unix shell as runtime requirement for octave. It'd 
>> > end up in a windows binary depending on a unix layer, and that's cygwin.
>> >
>> >> Suppose that libtool
>> >> were written in Perl.  Would you also object to including perl in the
>> >> distribution so that Octave could run the libtool script?
>> >
>> > Possibly. I have no experience with perl distributions for windows, so I 
>> > cannot estimate the amount of effort it would require to include it.
>> > I'd probably stick to a C++ version of mkoctfile anyway.
>> >
>> > This is also getting a bit philosophical.
>> >
>> > At the current stage, windows binaries of octave do not require any 
>> > privileges for installing a default user (who is not administrator) on 
>> > windows doesn't have.
>> > It even doesn't require being "installed" at all, you can just copy it and 
>> > run it off a usb stick, if you like. It doesn't need to fiddle with 
>> > windows registry and it can be cleanly uninstalled or removed by simply 
>> > deleting the directory tree it was installed to.
>> >
>> > Now beginning to add dependencies that don't follow this philosophy, may 
>> > mean that octave is no longer portable, and octave must strictly be 
>> > installed with administrative privileges. And this is IMO the wrong way to 
>> > go.
>> >
>> >> Octave
>> >> already includes a perl function.  Do we distribute a copy of Perl
>> >> with Octave on Windows systems so that the perl function can work?
>> >
>> > No, perl's not included. Now I'm curious. Which octave function is that?
>> >
>> > benjamin
>> > --
>>
>> The only perl script is octave/mk-opts.pl, and it is only used to
>> recreate the option handlers DASPK-opts and similar. I think that like
>> awk, bison, flex and gperf, perl is only needed to build from plain
>> repo, but not from the distribution tarball.
>>
>> If libtool was only required for generating distrib tarballs, it would
>> make little difference, but making it a requirement for mkoctfile
>> would mean a significant change. Not just for Windows; unless I'm
>> missing something, it would mean that dependency on libtool would be
>> required for Octave packages in GNU/Linux distros as well.
>
> Okay, from my viewpoint of Debian maintainer: I would jump for joy when
> Octave uses libtool.
>
> In fact, two or three years ago I looked into adding libtool to Octave,
> but I gave up when I realized that Octave didn't use automake (which is
> not strictly necessary, but the libtool manual only documents the
> automake use case).
>
> libtool is not some esoteric tool, but sort of standard for libraries in
> the Unix-like world. Using such a tool means you can ask on normal
> distribution lists and there will be literally hundreds of people with
> experience on this topic.  Whereas currently it means asking on the
> Octave lists and hoping that the few people here have an idea. Which, I
> might add, usually boils down to John in use cases like Solaris. Thus,
> I'd actually expect people to give him free reign in his decision or
> stand up and help the next time a question about Solaris comes up.
>
> Now, some things that came up in different mails in the thread, by
> different people:
>
> | For windows it simply doesn't make any sense to use it, just to get
> | source code compiled and linked. Creating an oct file is a one-liner
> | with the gnu compiler, and using a nearly-300K sized shell script to do
> | the job seems - well - strange?
>
> Yes, it takes one line. And it takes one line on Linux. And one line on
> Solaris. And one line on HP-UX.
> Unfortunately, these lines might differ on all of these systems. They
> might differ when using a different compiler. They might differ when
> using a different toolchain. Does anybody on this list know how to
> compile a shared library on VMS? I don't. But libtool does.
>
> http://www.fortran-2000.com/ArnaudRecipes/sharedlib.html
> to get an idea of the fiddling required.
>
> | If libtool was only required for generating distrib tarballs, it would
> | make little difference, but making it a requirement for mkoctfile
> | would mean a significant change. Not just for Windows; unless I'm
> | missing something, it would mean that dependency on libtool would be
> | required for Octave packages in GNU/Linux distros as well.
>
> I already had to patch mkoctfile to make it work reliable in Debian (it
> hardcodes paths when building Octave, which makes up for problems when
> the compiler is updated). So, no, mkoctfile is not a simple
> one-size-fits-all solution.
>
> Is libtool better? I don't know. But there are certainly more people
> with knowledge about libtool than there are with knowledge about
> mkoctfile and compiler flags.
>
> | With all due respect, but at this point I'd like to re-raise the
> | question, why are you trying to switch to libtool in the first place?
> | What's so complex about compiling a source file and linking a library
> | supposing for simplicity one is using the gnu compiler/linker suite?
> | (Does libtool even support different compilers?)
>
> I can only wonder that people argue about portability in one sentence,
> yet don't care about totally different compilers and platforms in the
> other sentences. You want portability? Use proven tools. And yes,
> libtool is complex and actually pretty slow when run.

This might be a drawback. How slow? Some packages require mkoctfile to
be run a dozen times to build ...

-- 
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz