Re: gub: I can now completely build lilypond

[Knut Petersen]

On 13.01.19 13:59, Werner LEMBERG wrote:
> Looks like it's `strace' time again...

Yes.

Building tools::python is broken on openSuSE tumbleweed as well as on ubuntu 
18, but inspection reveals that the problems are _not_ identical. On ubuntu 18 
the building of tools::python fails because libs are not found, but

   diff --git a/gub/specs/python.py b/gub/specs/python.py
   index 049b26f2..ac7251a0 100644
   --- a/gub/specs/python.py
   +++ b/gub/specs/python.py
   @@ -191,6 +191,11 @@ class Python__tools (tools.AutoBuild, Python):
             ]
         force_autoupdate = True
         parallel_build_broken = True
   +    configure_command = 
('LD_LIBRARY_PATH=%(system_prefix)s/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} '
   +                         + tools.AutoBuild.configure_command)
   +    install_command = 
('LD_LIBRARY_PATH=%(system_prefix)s/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} '
   +                         + tools.AutoBuild.install_command)
   +
         make_flags = Python.make_flags + ' LIBC="-lcrypt -ldb"'
         def patch (self):
             Python.patch (self)

helps to cure that (most other xxx::python builds are still broken).

On openSuSE tumbleweed the situation is different: Let's have  a look at a 
broken build:

   rm STRACE/*; strace -v -f -ff -s 1024 -o STRACE/TP bin/gub --fresh 
tools::python

fails with

   calculating dependencies
   Checking for g++ ... /usr/bin/g++
   Checking for gcc ... /usr/bin/gcc
   must rebuild[tools]: system::gcc system::g++ python
   building package: tools::python
     *** Stage: download (python, tools)
     *** Stage: untar (python, tools)
     *** Stage: patch (python, tools)
     *** Stage: autoupdate (python, tools)
     *** Stage: configure (python, tools)
     *** Stage: compile (python, tools)
   Command barfed: cd /home/gub/NewGub/gub/target/tools/build/python-2.4.5 && make   
BLDLIBRARY='-Wl,-rpath -Wl,\$$ORIGIN/../lib -Wl,-rpath 
-Wl,/home/gub/NewGub/gub/target/tools/root/usr/lib -L. -lpython$(VERSION)'  LIBC="-lcrypt 
-ldb"

   Tail of target/tools/log/python.log >>>>>>>>
            esac
        /bin/sh: line 1:  6376 Segmentation fault (core dumped) 
LD_LIBRARY_PATH=/home/gub/NewGub/gub/target/tools/build/python-2.4.5:/home/gub/NewGub/gub/target/tools/root
 CC='gcc -pthread -I/home/gub/NewGub/gub/target/tools/build/python-2.4.5' 
LDSHARED='gcc -pthread
   -I/home/gub/NewGub/gub/target/tools/build/python-2.4.5 -shared' 
OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E 
/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py build
        make: *** [sharedmods] Error 139
        Command barfed: cd /home/gub/NewGub/gub/target/tools/build/python-2.4.5 && make   
BLDLIBRARY='-Wl,-rpath -Wl,\$$ORIGIN/../lib -Wl,-rpath 
-Wl,/home/gub/NewGub/gub/target/tools/root/usr/lib -L. -lpython$(VERSION)'  LIBC="-lcrypt 
-ldb"
   <<<<<<<< Tail of target/tools/log/python.log

Segfault. Nice.

STRACE/TP.6376  shows the executable that caused the problem:

   execve("./python", ["./python", "-E", 
"/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py", "build"]

grep 'openat(' STRACE/TP.6376 | grep -v ENOENT' shows the files that were 
opened with success:

   openat(AT_FDCWD, 
"/home/gub/NewGub/gub/target/tools/root/usr/lib/librestrict.so", 
O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, 
"/home/gub/NewGub/gub/target/tools/build/python-2.4.5/libpython2.4.so.1.0", 
O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, 
"/home/gub/NewGub/gub/target/tools/root/usr/lib/libdb-4.7.so", 
O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, "/usr/lib64/libdb-4.8.so", O_RDONLY|O_CLOEXEC) = 3
   openat(AT_FDCWD, 
"/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py", O_RDONLY) = 3

Nothing suspicious with one exception: 
home/gub/NewGub/gub/target/tools/root/usr/lib/libdb-4.7.so is opened as well as 
/usr/lib64/libdb-4.8.so. Why did we link against libdb-4.8?

Let's have a look at an ubuntu 16.04 strace log of '/python -E 
/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py':

Rebuild with strace, find the relevant log:

   mkdir STRACE
   rm -f STRACE/*; strace -v -f -ff -s 1024 -o STRACE/TP bin/gub --fresh 
tools::python
   grep '"-E", "/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py", "build"' 
STRACE/* | sed -e "s/\([^:]*\)[[:print:]]*/\1/"

reveals two log files. setup.py is processed twice. Obviously the older is the 
file to inspect. Which files are opened during the successful build on ubuntu?

   grep 'open(' STRACE/TP.17633 | grep -v ENOENT

gives

   open("/home/gub/NewGub/gub/target/tools/root/usr/lib/librestrict.so", 
O_RDONLY|O_CLOEXEC) = 3
   
open("/home/gub/NewGub/gub/target/tools/build/python-2.4.5/libpython2.4.so.1.0",
 O_RDONLY|O_CLOEXEC) = 3
   open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
   open("/home/gub/NewGub/gub/target/tools/root/usr/lib/libdb-4.7.so", 
O_RDONLY|O_CLOEXEC) = 3
   open("/lib/i386-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
   open("/home/gub/NewGub/gub/target/tools/src/python-2.4.5/setup.py", 
O_RDONLY|O_LARGEFILE) = 3
   open("/home/gub/NewGub/gub/target/tools/src/python-2.4.5/Lib/site.py", 
O_RDONLY|O_LARGEFILE) = 4
   [... lots of files ...]

Here our own libdb-4.7.so is used as expected, not libdb* from /usr/lib*.

No time left for today ... maybe someone else has an idea how to prevent 
tooly::python from using the system's libdb-4.8.

Knut