BLT2.4 packaging troubles in Ubuntu (was Re: [swarm-hackers] swarm 2.3.0 release

[Paul Johnson]

I'm starting this as a new thread so we can keep focused and because it 
is starting to really piss me off.

Scott Christley wrote:
> I installed a completely new ubuntu 8.10 along with tcl/tk 8.4 and the  
> blt-dev package.  Swarm fails to compile as it cannot find tcl8.5, so  
> essentially the blt package on ubuntu does not support tcl8.4.  I  
> suppose maybe this was a packaging decision, we could file a ubuntu  
> bug and ask them to provide a 8.4 compatible package, or provide one  
> ourselves.
>
>   
I don't presume to know about programming like you do, but I'm hell on 
wheels when it comes to pointing and clicking in Synaptic :)

In Ubuntu 8.10, you can choose whether to install packages tk/tcl, or 
tk8.4/tcl8.4, and/or tk8.5/tcl8.5.
If you install the tk8.4-dev and tk8.5-dev packages along with blt-dev, 
then you are supposed to be able to compile blt programs either with 
tcl/tk 8.4 or 8.5. 

blt-dev enforces a requirement that you have tcl8.4-dev or tcl8.5-dev, 
but not both. I believe it would tolerate both, however.

With only the tcl8.5-dev , tk8.5-dev and blt-dev packages installed, I 
was able to compile & install Swarm, but I got that first seg fault I 
was whining about the other day.

But the tcl/tk8.4 blt package from Hardy does work.  The Intrepid hybrid 
packaging is  breaking something. I see lots of possible mistakes here, 
please bear with me.

CANDIDATE 1. SYMBOLIC LINKS in /usr/lib

When they install the shared and static libraries, there are two 
versions, named like:

libBLT.2.4.so.8.4

libBLT.2.4.so.8.5

they create a symbolic link so that

/usr/lib/libBLT.so -> libBLT.2.4.so.8.5

Well, hell, that screws up Swarm programs bigtime. We have -lBLT in the 
extra libs for programs. So Supposing  you configure Swarm 
--with-tcl=/usr/lib/tcl8.4 and all seems well, then the compile of Swarm 
apps links in both version of blt libraries.  End result, crashola.

That may also explain why "ldd ./heatbugs" showed both tcl/tk 8.4 and 
8.5, even though Swarm itself was compiled against 8.4.

I tried the easy fix-change the libBLT.so symbolic link point at 
libBLT.2.4.so.8.4.  I've tested that, rebuilt Swarm.  Still crashes, and 
the "ldd" output still shows the linker is grabbing more than one 
edition of tcl/tk.  But maybe I did not test it sufficiently enough.  Or 
properly.  (One danger of Debian packaging is that you are allowed to 
muck about in the source code without any accountability...)

CANDIDATE 2: Other FLAWS IN PACKAGING

I've got the source code and patches applied for building packages for 
blt  in Intrepid (8.10) and Hardy (8.04).  Here's the essence of their 
approach.  Try to create versions of the blt "a" and "so" libs for two 
versions of tcl.tk.  However, I do not have any evidence that this has 
actually ever worked for any living human.  In 8.04, they supposedly 
supported tcl/tk8.3 and tcl/tk8.4, but we (and other living humans I 
know) were only using 8.4.  I don't have any evidence that it was 
effective to compile any program (incl. Swarm) against tcl/tk 8.3 with 
blt2.4z. Now, I believe in Ubuntu 8.10, there is evidence that tcl/tk8.5 
will work with blt, but I don't have any evidence of tcl/tk8.4 working.

I've been reading the packaging code and here is how it is done. Unlike 
RedHat packaging, there is no single file that determines what happens. 
Instead, big patch is applied to the source and it creates a 
subdirectory inside blt-2.4z called "debian."  In the debian/rules file, 
the Makefile for the packaging, here's the part that builds the 2 sets 
of libraries for blt:

#!/usr/bin/make -f
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
# This has to be exported to make some magic below work.
export DH_OPTIONS


# useful variables
v = 2.4

dtmp = $(shell pwd)/debian/tmp

CONFIGURE = ./configure --prefix=/usr/ --mandir=/usr/share/man \
   --with-cflags="-O2 -g -D_REENTRANT"

# Now, the build targets...

build: build-stamp
build-stamp: build-8.4-stamp build-8.5-stamp
   touch build-stamp

#
# build for tcl/tk version (8.4)
#
build-8.4-stamp:
   dh_testdir
   -mkdir -p tmplib

   -$(MAKE) distclean
   rm -rf debian/tcl8.4 debian/tk8.4

   -mkdir -p debian/tcl8.4/include debian/tk8.4/include
   -mkdir -p debian/tcl8.4/lib debian/tk8.4/lib

   ln -s /usr/lib/libtcl8.4.so.1 debian/tcl8.4/lib/libtcl8.4.so
   ln -s /usr/lib/libtk8.4.so.1  debian/tk8.4/lib/libtk8.4.so

   cp debian/tcl.h.tcl8.4        debian/tcl8.4/include/tcl.h
   cp debian/tclDecls.h.tcl8.4   debian/tcl8.4/include/tclDecls.h
   cp debian/tclPlatDecls.h.tcl8.4   debian/tcl8.4/include/tclPlatDecls.h
   cp debian/tclConfig.sh.tcl8.4 debian/tcl8.4/tclConfig.sh
   echo TCL_PREFIX="'"`pwd`/debian/tcl8.4"'" \
       >> debian/tcl8.4/tclConfig.sh
   echo TCL_EXEC_PREFIX="'"`pwd`/debian/tcl8.4"'" \
       >> debian/tcl8.4/tclConfig.sh

   cp debian/tk.h.tk8.4        debian/tk8.4/include/tk.h
   cp debian/tkDecls.h.tk8.4   debian/tk8.4/include/tkDecls.h
   cp debian/tkPlatDecls.h.tk8.4   debian/tk8.4/include/tkPlatDecls.h
   cp debian/tkConfig.sh.tk8.4 debian/tk8.4/tkConfig.sh
   echo TK_PREFIX="'"`pwd`/debian/tk8.4"'" \
       >> debian/tk8.4/tkConfig.sh
   echo TK_EXEC_PREFIX="'"`pwd`/debian/tk8.4"'" \
       >> debian/tk8.4/tkConfig.sh

   $(CONFIGURE) --with-tk=debian/tk8.4 --with-tcl=debian/tcl8.4

   ( cd src/shared ; \
     sed -e 's:$$(version)$$(SHLIB_SUFFIX):.2.4.so.8.4:' Makefile > M2 ; \
     mv M2 Makefile )

   ( cd src; $(MAKE) )
   mv src/shared/libBLTlite* tmplib/libBLTlite.$(v).so.8.4
   mv src/shared/libBLT*     tmplib/libBLT.$(v).so.8.4
   mv src/libBLTlite.a       tmplib/libBLTlite.$(v).8.4.a
   mv src/libBLT.a           tmplib/libBLT.$(v).8.4.a

   touch build-8.4-stamp

#
# Now build for the best tcl/tk version (8.5)
#
build-8.5-stamp:
   -$(MAKE) distclean

   $(CONFIGURE) --with-tk=/usr/lib/tk8.5 --with-tcl=/usr/lib/tcl8.5

   ( cd src/shared ; \
     sed -e 's:$$(version)$$(SHLIB_SUFFIX):.2.4.so.8.5:' Makefile > M2 ; \
     mv M2 Makefile )

   $(MAKE) all

   touch build-8.5-stamp
==================snip============================


In the part for tcl8.5, note it is standard. The configure uses the 
installed libraries & headers.

In the 8.4 section, well, it is a weird mix.  The patch provides tcl.h 
tk.h header files, but they use symbolic links to get the shared 
libraries from the installed tcl/tk8.4. I believe it is like this so you 
can build blt if you have tcl/tk8.4 installed WITHOUT the devel 
packages. They copy their headers into a directory of the build 
"debian/tcl8.4" and then when they run the configure, they point 
--with-tcl at that directory.

I believe this is the "mismatch" problem Marcus was describing today, 
where a very subtle difference in library editions will break everything 
all to hell.  Note the rules assumes:

 ln -s /usr/lib/libtcl8.4.so.1 debian/tcl8.4/lib/libtcl8.4.so

But on my Intrepid system, there is no such shared library. Instead, I 
have so.0:

$ ls -la /usr/lib/libtcl*
-rw-r--r-- 1 root root 1167656 2008-06-23 09:23 /usr/lib/libtcl8.4.a
lrwxrwxrwx 1 root root      14 2009-04-07 20:30 /usr/lib/libtcl8.4.so -> 
libtcl8.4.so.0
-rw-r--r-- 1 root root  754896 2008-06-23 09:23 /usr/lib/libtcl8.4.so.0
-rw-r--r-- 1 root root 1046212 2008-07-04 03:25 /usr/lib/libtcl8.5.so.0
-rw-r--r-- 1 root root    2202 2008-06-23 09:23 /usr/lib/libtclstub8.4.a

So I think this means that, if I am using the blt they provide, and I 
compile something against tcl/tk8.4, it will compile, but when it is 
run, it will crash, because the whole thing assumes I've got the same 
libtcl8.4.so.1 that they had.  I can't imagine any way in which that 
would work.

Their packaging in Hardy followed that exact same approach, except that 
it used the system's version of tcl/tk for 8.4 and it used the copied in 
versions it supplied for 8.3.


IF I believed that their packaging approach for blt with 2 versions of 
tcl/tk were valid, then maybe I could fix it by correcting the versions 
and re-building blt.  that was my first guess, but after re-building, I 
had same trouble. But it could easily be that there is some other 
mistake in their packaging that I've not spotted yet.

They are using the same BLT source code in both cases.  So we know we 
"should" be able to make Swarm work with tcl/tk 8.4 installed. 

The million dollar question is, can we ever make Swarm work against blt  
w/tcl/tk8.4 on a system that is also trying to support tcl/tk8.5. 

I'm wondering why a person would not just build 2 blt packages, one for 
tcl/tk8.4 and one for tcl/tk8.5.  It seems like there would be precedent 
for packages blt8.4 and blt8.5.

But I do not know of a way to test the ability to compile ANY tcl/tk 8.4 
program with blt2.5.



--
Paul E. Johnson                       email: address@hidden
Professor, Political Science          http://pj.freefaculty.org
1541 Lilac Lane, Rm 504                              
University of Kansas                  Office: (785) 864-9086
Lawrence, Kansas 66044-3177           FAX: (785) 864-5700