Re: [Qemu-devel] undefined symbol _nettle_cast5_set_key

[Daniel P. Berrange]

On Tue, May 31, 2016 at 10:25:20AM -0400, Programmingkid wrote:
> 
> On May 31, 2016, at 10:00 AM, Daniel P. Berrange wrote:
> 
> > On Fri, May 20, 2016 at 04:43:51PM -0600, Eric Blake wrote:
> >> On 05/20/2016 01:15 PM, Programmingkid wrote:
> >>> Noticed this bug after a 'git pull'. My repo is at commit 
> >>> 65603e2fc18b48e6e55a3dd693669413141694ec - 
> >>> tci: do not include exec/exec-all.h
> >>> 
> >>> I am building QEMU on Mac OS 10.6 with GCC 4.9.
> >>> Here is the full message:
> >>> 
> >>> LINK  qemu-nbd
> >>> Undefined symbols for architecture x86_64:
> >>>  "_nettle_cast5_set_key", referenced from:
> >>>      _qcrypto_cipher_new in cipher.o
> >> 
> >> What version of nettle are you compiling against? Obviously it's an
> >> older version, but is it one that we need to work around at configure
> >> time, or one that you should just update locally?
> > 
> > So we already have compatibility code to allow us to build with older
> > versions of nettle that I've tested right back to nettle 2.4.
> > 
> > The "cast5_set_key" method is present in the nettle 3.2 version that
> > Programmingkid has installed. The nettle headers mangle the name
> > 'cast5_set_key' into 'nettle_cast5_set_key', but I'm not sure where
> > the extra leading '_' is coming from in either _nettle_cast5_set_key
> > or _qcrypto_cipher_new. Presumably that is something specific to
> > Os-X linking.
> > 
> > So I don't think this is a question of incompatible older versions of
> > nettle. Rather than is some problem with linking going on here.
> > 
> > 
> > I'd like to see the output of 'configure' as well as the config.log
> > and config-host.mak files that it generates when showing this error.
> > 
> > Also I'd like the output of 'make V=1' so we see the full linker
> > command line, not just the summary.
> > 
> > Regards,
> > Daniel
> 
> Here is everything you wanted: configure output, config.log, config-host.mak, 
> and make V=1.
> 
> $ ./configure --cxx=gcc-4.9 --cc=gcc-4.9 --objcc=gcc-4.9 --disable-gtk 
> --disable-sdl --target-list=ppc-softmmu,i386-softmmu
> ld: unknown option: --verbose


> GNUTLS support    yes
> GNUTLS hash       yes
> GNUTLS rnd        yes
> libgcrypt         no
> libgcrypt kdf     no
> nettle            yes (3.2)
> nettle kdf        yes

Ok, that shows we've detected your nettle 3.2 install. The
support for 'kdf' also shows we successfully compiled and
linked against that. This is shows your 3.2 install of nettle
is present & correct which is good.

> libtasn1          yes
> curses support    yes
> virgl support     no
> curl support      yes
> mingw32 support   no
> Audio drivers     coreaudio
> Block whitelist (rw) 
> Block whitelist (ro) 
> VirtFS support    no
> VNC support       yes
> VNC SASL support  yes
> VNC JPEG support  yes
> VNC PNG support   yes
> xen support       no
> brlapi support    no
> bluez  support    no
> Documentation     yes
> PIE               no
> vde support       no
> netmap support    no
> Linux AIO support no
> ATTR/XATTR support no
> Install blobs     yes
> KVM support       no
> RDMA support      no
> TCG interpreter   no
> fdt support       yes
> preadv support    no
> fdatasync         no
> madvise           yes
> posix_madvise     yes
> sigev_thread_id   no
> uuid support      yes
> libcap-ng support no
> vhost-net support no
> vhost-scsi support no
> Trace backends    log
> spice support     no 
> rbd support       no
> xfsctl support    no
> smartcard support no
> libusb            yes
> usb net redir     yes
> OpenGL support    no
> OpenGL dmabufs    no
> libiscsi support  no
> libnfs support    no
> build guest agent yes
> QGA VSS support   no
> QGA w32 disk info no
> QGA MSI support   no
> seccomp support   no
> coroutine backend sigaltstack
> coroutine pool    yes
> GlusterFS support no
> Archipelago support no
> gcov              gcov
> gcov enabled      no
> TPM support       yes
> libssh2 support   no
> TPM passthrough   no
> QOM debugging     yes
> vhdx              yes
> lzo support       no
> snappy support    no
> bzip2 support     yes
> NUMA host support no
> tcmalloc support  no
> jemalloc support  no
> avx2 optimization no
> 
> 
> 
> The config.log file:
> 


> gcc-4.9 -m64 -DOS_OBJECT_USE_OBJC=0 -arch x86_64 -D_GNU_SOURCE 
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
> -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs 
> -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self 
> -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition 
> -Wtype-limits -fstack-protector-strong 
> -I/usr/local/Cellar/gnutls/3.4.11/include 
> -I/usr/local/Cellar/nettle/3.2/include 
> -I/usr/local/Cellar/libtasn1/4.8/include -I/opt/local/include -o 
> config-temp/qemu-conf.exe config-temp/qemu-conf.c -m64 -framework 
> CoreFoundation -framework IOKit -arch x86_64 -g 
> -L/usr/local/Cellar/gnutls/3.4.11/lib -lgnutls



> gcc-4.9 -m64 -DOS_OBJECT_USE_OBJC=0 -arch x86_64 -D_GNU_SOURCE 
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
> -fno-strict-aliasing -fno-common -Wendif-labels -Wmissing-include-dirs 
> -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self 
> -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition 
> -Wtype-limits -fstack-protector-strong 
> -I/usr/local/Cellar/gnutls/3.4.11/include 
> -I/usr/local/Cellar/nettle/3.2/include 
> -I/usr/local/Cellar/libtasn1/4.8/include -I/opt/local/include 
> -I/usr/local/Cellar/nettle/3.2/include -I/usr/local/Cellar/nettle/3.2/include 
> -o config-temp/qemu-conf.exe config-temp/qemu-conf.c -m64 -framework 
> CoreFoundation -framework IOKit -arch x86_64 -g 
> -L/usr/local/Cellar/nettle/3.2/lib -lnettle

This is the link line where configure tested for the KDF function and it looks 
normal and
is pointing the linker to /usr/local/Cellar/nettle/3.2/lib



> gcc-4.9 -I/opt/local/include/pixman-1 
> -I/Users/john/Documents/Development/Projects/Qemu/qemu-git/dtc/libfdt -m64 
> -DOS_OBJECT_USE_OBJC=0 -arch x86_64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 
> -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef 
> -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common  
> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs 
> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers 
> -Wold-style-declaration -Wold-style-definition -Wtype-limits 
> -fstack-protector-strong -I/usr/local/Cellar/gnutls/3.4.11/include 
> -I/usr/local/Cellar/nettle/3.2/include 
> -I/usr/local/Cellar/libtasn1/4.8/include -I/opt/local/include 
> -I/usr/local/Cellar/nettle/3.2/include   -I/opt/local/include/libpng16 
> -I/usr/local/Cellar/libusb/1.0.20/include/libusb-1.0 
> -I/usr/local/Cellar/usbredir/0.7.1/include 
> -I/Users/john/Documents/Development/Projects/Qemu/qemu-git/tests -O2 
> -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_REENTRANT 
> -I/usr/local/Cellar/glib/2.46.2/include/glib-2.0 
> -I/usr/local/Cellar/glib/2.46.2/lib/glib-2.0/include 
> -I/usr/local/opt/gettext/include -g  -m64 -framework CoreFoundation 
> -framework IOKit -arch x86_64 -g  -o qemu-nbd qemu-nbd.o async.o 
> thread-pool.o block.o blockjob.o main-loop.o iohandler.o qemu-timer.o 
> aio-posix.o qemu-io-cmds.o block/raw_bsd.o block/qcow.o block/vdi.o 
> block/vmdk.o block/cloop.o block/bochs.o block/vpc.o block/vvfat.o 
> block/qcow2.o block/qcow2-refcount.o block/qcow2-cluster.o 
> block/qcow2-snapshot.o block/qcow2-cache.o block/qed.o block/qed-gencb.o 
> block/qed-l2-cache.o block/qed-table.o block/qed-cluster.o block/qed-check.o 
> block/vhdx.o block/vhdx-endian.o block/vhdx-log.o block/quorum.o 
> block/parallels.o block/blkdebug.o block/blkverify.o block/blkreplay.o 
> block/block-backend.o block/snapshot.o block/qapi.o block/raw-posix.o 
> block/null.o block/mirror.o block/io.o block/throttle-groups.o block/nbd.o 
> block/nbd-client.o block/sheepdog.o block/accounting.o block/dirty-bitmap.o 
> block/write-threshold.o block/crypto.o nbd/server.o nbd/client.o nbd/common.o 
> block/curl.o block/dmg.o crypto/init.o crypto/hash.o crypto/aes.o 
> crypto/desrfb.o crypto/cipher.o crypto/tlscreds.o crypto/tlscredsanon.o 
> crypto/tlscredsx509.o crypto/tlssession.o crypto/secret.o 
> crypto/random-gnutls.o crypto/pbkdf.o crypto/pbkdf-nettle.o crypto/ivgen.o 
> crypto/ivgen-essiv.o crypto/ivgen-plain.o crypto/ivgen-plain64.o 
> crypto/afsplit.o crypto/xts.o crypto/block.o crypto/block-qcow.o 
> crypto/block-luks.o io/channel.o io/channel-buffer.o io/channel-command.o 
> io/channel-file.o io/channel-socket.o io/channel-tls.o io/channel-watch.o 
> io/channel-websock.o io/channel-util.o io/task.o qom/object.o qom/container.o 
> qom/qom-qobject.o qom/object_interfaces.o  libqemuutil.a libqemustub.a  -lz 
> -L/opt/local/lib -lcurl -lbz2 -L/usr/local/Cellar/glib/2.46.2/lib 
> -L/usr/local/opt/gettext/lib -lgthread-2.0 -lglib-2.0 -lintl  -lz -lz 
> -L/usr/local/Cellar/nettle/3.2/lib -lnettle 
> -L/usr/local/Cellar/gnutls/3.4.11/lib -lgnutls  -lutil
> Undefined symbols for architecture x86_64:
>   "_nettle_cast5_set_key", referenced from:
>       _qcrypto_cipher_new in cipher.o
> ld: symbol(s) not found for architecture x86_64
> collect2: error: ld returned 1 exit status
> make: *** [qemu-nbd] Error 1

This linker line again looks basically sound. It is still passing the
correct -lnettle arg, and the -I and -L args to point to the nettle
3.2 installation.

Given that we found the "KDF" support in nettle, we know that it is
new enough to support the "nettle_cast5_set_key" function too so
we ought to link correctly.

The only thing that occurs to me is that we have a number of other
-L args present - eg -L/opt/local/lib.

I wonder if one of those  other search paths has a differnet version
of nettle installed in it, causing gcc to link to the wrong one.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|