Re: 3 failed tests for libidn2-2.1.1

[Tim Rühsen]

Hi Dennis,

2.1.1a vs 2.1.1: It's the same source. Just the 2.1.1 tarball missed a
man page due to an unrecognized error (missing ronn utility). So I
uploaded a 2.1.1a tarball that includes the man page.
Sorry for the inconvenience, next time I'll make up a complete new
release - you are not the only one who was/is confused.

I can't reproduce the issues seen in test-lookup.log on a OpenCSW
Solaris box:
uname -m = i86pc
uname -r = 5.11
uname -s = SunOS
uname -v = 11.3

Other differences are:
- using gcc 5.5.0
- GNU environment
- using latest git master

~/libidn2 > ldd src/.libs/idn2
        libidn2.so.0 =>  /opt/csw/lib/libidn2.so.0
        libiconv.so.2 =>         /opt/csw/lib/libiconv.so.2
        libintl.so.9 =>  /opt/csw/lib/libintl.so.9
        libunistring.so.2 =>     /opt/csw/lib/libunistring.so.2
        libc.so.1 =>     /lib/libc.so.1
        libgcc_s.so.1 =>         /opt/csw/lib/i386/libgcc_s.so.1

From config.log:
  Version:           2.1.1
  Host OS:           solaris2.11
  Install prefix:    /usr/local
  Compiler:          gcc
  Warning flags:      -fno-common -W -Waddress
-Waggressive-loop-optimizations -Wall -Wattributes -Wbad-function-cast
-Wbool-compare -Wbuiltin-macro-redefined -Wcast-align -Wchar-subscripts
-Wclobbered -Wcomment -Wcomments -Wcoverage-mismatch -Wcpp -Wdate-time
-Wdeprecated -Wdeprecated-declarations -Wdesignated-init
-Wdisabled-optimization -Wdiscarded-array-qualifiers
-Wdiscarded-qualifiers -Wdiv-by-zero -Wdouble-promotion -Wempty-body
-Wendif-labels -Wenum-compare -Wextra -Wformat-contains-nul
-Wformat-extra-args -Wformat-nonliteral -Wformat-security
-Wformat-signedness -Wformat-y2k -Wformat-zero-length
-Wfree-nonheap-object -Wignored-qualifiers -Wimplicit
-Wimplicit-function-declaration -Wimplicit-int
-Wincompatible-pointer-types -Winit-self -Winline -Wint-conversion
-Wint-to-pointer-cast -Winvalid-memory-model -Winvalid-pch
-Wlogical-not-parentheses -Wlogical-op -Wmain -Wmaybe-uninitialized
-Wmemset-transposed-args -Wmissing-braces -Wmissing-declarations
-Wmissing-include-dirs -Wmissing-parameter-type -Wmissing-prototypes
-Wmultichar -Wnarrowing -Wnested-externs -Wnonnull -Wodr
-Wold-style-declaration -Wold-style-definition -Wopenmp-simd -Woverflow
-Woverlength-strings -Woverride-init -Wpacked -Wpacked-bitfield-compat
-Wparentheses -Wpointer-arith -Wpointer-sign -Wpointer-to-int-cast
-Wpragmas -Wpsabi -Wreturn-local-addr -Wreturn-type -Wsequence-point
-Wshadow -Wshift-count-negative -Wshift-count-overflow
-Wsizeof-array-argument -Wsizeof-pointer-memaccess -Wstack-protector
-Wstrict-aliasing -Wstrict-overflow -Wstrict-prototypes
-Wsuggest-attribute=const -Wsuggest-attribute=format
-Wsuggest-attribute=noreturn -Wsuggest-attribute=pure
-Wsuggest-final-methods -Wsuggest-final-types -Wswitch -Wswitch-bool
-Wsync-nand -Wtrampolines -Wtrigraphs -Wtype-limits -Wuninitialized
-Wunknown-pragmas -Wunsafe-loop-optimizations -Wunused
-Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-function
-Wunused-label -Wunused-local-typedefs -Wunused-macros
-Wunused-parameter -Wunused-result -Wunused-value -Wunused-variable
-Wvarargs -Wvariadic-macros -Wvector-operation-performance -Wvla
-Wvolatile-register-var -Wwrite-strings -Warray-bounds=2
-Wnormalized=nfc -fdiagnostics-show-option -fdiagnostics-color=always
-Wno-missing-field-initializers
  CFLAGS:            -g -O2
  LDFlags:
  Documentation:     no
  Library types:     Shared=yes, Static=yes
  Libiconv:          yes -liconv
  Libunistring:      yes 0.9.9 -lunistring
  Valgrind:
  Version script:    no


Now, let's do some guessing / eliminating:
What happens when you change
-  FILE *fp = fopen(fname, "r");
+  FILE *fp = fopen(fname, "rb");
in tests/test-lookup.c and re-run the test ?

Do you see the same output as I do ?
~/libidn2 > src/.libs/idn2 -d xn--fze90fu1iw8r.1|od -t x1
0000000 e1 a2 a2 e1 9c 94 e1 b7 a6 e2 b4 87 2e 31 0a
0000017

~/libidn2 > src/.libs/idn2 -d xn--fze90fu1iw8r.1|src/.libs/idn2
xn--fze90fu1iw8r.1

This was one of your failing tests.

Regards, Tim

On 5/13/19 4:59 AM, Dennis Clarke wrote:
> 
> 
> Thank you for the reply Tim.
> 
> Sometimes progres is made just by looking more closely and then trying
> to explain to someone else.  Wherein I saw the pre-existing libs from
> last year.
> 
>>> gmake[4]: Entering directory
>>> '/usr/local/build/libidn2-2.1.1a_SunOS5.10_sparc64vii+.001/tests'
>>> PASS: test-punycode
>>> FAIL: test-lookup
>>> PASS: test-register
>>> PASS: test-strerror
>>> PASS: test-tounicode
>>> ============================================================================
>>>
>>>
>>> Testsuite summary for libidn2 2.1.1
>>> ============================================================================
>>>
>>>
>>> # TOTAL: 5
>>> # PASS:  4
>>> # SKIP:  0
>>> # XFAIL: 0
>>> # FAIL:  1
>>> # XPASS: 0
>>> # ERROR: 0
>>> ============================================================================
>>>
>>>
>>> See tests/test-suite.log
>>> Please report to address@hidden
>>> ============================================================================
>>>
>>>
>>>
>>> OKay that is better.
>>>
>>> Still a failure there of course.
>>
>> If test-lookup still has the same failure as in your first post: I was
>> wrong, idn2_to_ascii_4i2() is of course not deprecated
>> (idn2_to_ascii_4i() is).
>>
>> That function is new since 2.1.0.
>>
> 
> This is 2.1.1a however the tarball extracts as just 2.1.1.
> 
> Looking into the /usr/local/include/idn2.h header I see :
> 
> #define IDN2_VERSION_NUMBER 0x02010001
> #define IDN2_VERSION_MAJOR 2
> #define IDN2_VERSION_MINOR 1
> #define IDN2_VERSION_PATCH 1
> 
> I guess I could just make the call to :
> 
> NAME
>      idn2_check_version - API function
> 
> SYNOPSIS
>      #include <idn2.h>
> 
>      const char * idn2_check_version(const char * req_version);
> 
> ARGUMENTS
>      const char * req_version
>                  version string to compare with, or NULL.
> 
> 
> beta$ cat /tmp/iv2.c
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <idn2.h>
> int main(int argc,char *argv[])
> {
>     printf("idn2_check_version(IDN2_VERSION) returns %s\n",
>                              idn2_check_version(IDN2_VERSION));
>     return EXIT_SUCCESS;
> }
> 
> beta$ $CC $CFLAGS -I/usr/local/include -L/usr/local/lib -o /tmp/iv2
> /tmp/iv2.c -lidn2
> beta$ /tmp/iv2
> idn2_check_version(IDN2_VERSION) returns 2.1.1
> beta$
> beta$ ldd /tmp/iv2
>         libidn2.so.0 =>  /usr/local/lib/libidn2.so.0
>         libc.so.1 =>     /lib/64/libc.so.1
>         libintl.so.7 =>  /usr/local/lib/libintl.so.7
>         libiconv.so.2 =>         /usr/local/lib/libiconv.so.2
>         libunistring.so.2 =>     /usr/local/lib/libunistring.so.2
>         libm.so.2 =>     /lib/64/libm.so.2
>         /lib/sparcv9/../libm/sparcv9/libm_hwcap1.so.2
>         /platform/SUNW,SPARC-Enterprise/lib/sparcv9/libc_psr.so.1
> beta$
> 
> So I guess the 'a' is something akin to IDN2_VERSION_PATCH but lessor.
> Regardless let's dig here a bit.
> 
>> So please look into test-lookup.log what it says now. Is there a 'ldd'
>> command to show the linked libraries ? Does libidn2.so.0 point to the
>> correct library file ?
> 
> beta$
> beta$ file ./tests/test-lookup
> ./tests/test-lookup: ELF 64-bit MSB executable SPARCV9 Version 1,
> dynamically linked, not stripped
> beta$ ldd ./tests/test-lookup
>         libidn2.so.0 =>  /usr/local/lib/libidn2.so.0
>         libintl.so.7 =>  /usr/local/lib/libintl.so.7
>         libunistring.so.2 =>     /usr/local/lib/libunistring.so.2
>         libiconv.so.2 =>         /usr/local/lib/libiconv.so.2
>         libc.so.1 =>     /lib/64/libc.so.1
>         libm.so.2 =>     /lib/64/libm.so.2
>         /lib/sparcv9/../libm/sparcv9/libm_hwcap1.so.2
>         /platform/SUNW,SPARC-Enterprise/lib/sparcv9/libc_psr.so.1
> beta$
> beta$ elfdump -devl ./tests/test-lookup | grep -E -e 'NEED|PATH'
>        [0]  NEEDED            0x26a               libidn2.so.0
>        [1]  NEEDED            0x277               libintl.so.7
>        [2]  NEEDED            0x284               libunistring.so.2
>        [3]  NEEDED            0x296               libiconv.so.2
>        [4]  NEEDED            0x257               libc.so.1
>        [7]  RUNPATH           0x2a4
> /usr/local/lib:/usr/local/build/libidn2-2.1.1a_SunOS5.10_sparc64vii+.002/lib/.libs
> 
>        [8]  RPATH             0x2a4
> /usr/local/lib:/usr/local/build/libidn2-2.1.1a_SunOS5.10_sparc64vii+.002/lib/.libs
> 
>       [15]  VERNEED           0x100000d88
>       [16]  VERNEEDNUM        0x1
> beta$
> 
> That RUNPATH and RPATH should have the local build directory first in
> the list for testing. That will certainly cause issues on a system with
> a previous version around. Which is what I had. I see the abomination
> known as LD_LIBRARY_PATH to be only used in oddball cases.  It works for
> doing testing of a new lib however the rule is that the ELF files should
> always have the RPATH/RUNPATH data in place correctly.
> 
> Some of that is just my crusty opinion but it works!  :)
> 
> In any case I did the build and test cycle twice and in both cases I
> also did the install. Of course on the second pass we get better results
> because now the correct version is in place within /usr/local/lib and
> we also have them in the build directory tree.  Good stuff.
> 
> So we know the version is correct given that the correct version is now
> located everywhere we would want it :
> 
> beta$
> beta$ ls -lapb ./lib/.libs/libidn2*
> -rw-r--r--   1 dclarke  devl      537072 May 12 16:11 ./lib/.libs/libidn2.a
> -rw-r--r--   1 dclarke  devl         337 May 12 16:11
> ./lib/.libs/libidn2.exp
> lrwxrwxrwx   1 dclarke  devl          13 May 12 16:11
> ./lib/.libs/libidn2.la -> ../libidn2.la
> -rw-r--r--   1 dclarke  devl        1099 May 12 16:11
> ./lib/.libs/libidn2.lai
> lrwxrwxrwx   1 dclarke  devl          16 May 12 16:11
> ./lib/.libs/libidn2.so -> libidn2.so.0.3.5*
> lrwxrwxrwx   1 dclarke  devl          16 May 12 16:11
> ./lib/.libs/libidn2.so.0 -> libidn2.so.0.3.5*
> -rwxr-xr-x   1 dclarke  devl      396480 May 12 16:11
> ./lib/.libs/libidn2.so.0.3.5
> 
> 
> Great.
> 
> Also :
> 
> beta$ ldd ./src/.libs/idn2
>         libidn2.so.0 =>  /usr/local/lib/libidn2.so.0
>         libintl.so.7 =>  /usr/local/lib/libintl.so.7
>         libunistring.so.2 =>     /usr/local/lib/libunistring.so.2
>         libiconv.so.2 =>         /usr/local/lib/libiconv.so.2
>         libc.so.1 =>     /lib/64/libc.so.1
>         libm.so.2 =>     /lib/64/libm.so.2
>         /lib/sparcv9/../libm/sparcv9/libm_hwcap1.so.2
>         /platform/SUNW,SPARC-Enterprise/lib/sparcv9/libc_psr.so.1
> beta$
> beta$ ls -alpb /usr/local/lib/libidn2*
> -rw-r--r--   1 root     root      537064 May 12 16:02
> /usr/local/lib/libidn2.a
> -rwxr-xr-x   1 root     root        1099 May 12 16:02
> /usr/local/lib/libidn2.la
> lrwxrwxrwx   1 root     root          16 May 12 16:02
> /usr/local/lib/libidn2.so -> libidn2.so.0.3.5*
> lrwxrwxrwx   1 root     root          16 May 12 16:02
> /usr/local/lib/libidn2.so.0 -> libidn2.so.0.3.5*
> -rwxr-xr-x   1 root     root      396488 May 12 16:02
> /usr/local/lib/libidn2.so.0.3.5
> beta$
> 
> So that is fine.
> 
> 
> beta$
> beta$ wc -l ./tests/test-lookup.log
>    11883 ./tests/test-lookup.log
> beta$
> 
> So that is large enough I can compress it and attach here.
> 
> Most of it is unreadable to me. Modem noise is more clear ;)
> 
> However :
> 
> 
> ##########N#\u0301#M-^L#xn--nxasmm1c##
> Failed: _check_toASCII(M-LM-^A) -> -303 (expected 0) 0
> 
> Failed: _check_toASCII(address@hidden) -> a.b.c.d (expected a.b.c.d.)
> 10012af30
> 
> Failed: _check_toASCII(M-fM-^WM-%M-fM-^\M-,) -> xn--wgv71a (expected
> xn--wgv71a119e.jp) 10012af10
> u8_to_u32(M-fM-^WM-%M-fM-^\M-,address@hidden<) failed (88)
> 
> Failed: _check_toASCII() ->  (expected xn--53h) 10012af30
> 
> Failed:
> _check_toASCII(address@hidden<M-^MM-aM-
> M-^LM-bM-^DM-,address@hidden
> M-^TM-0M-sM- M-^GM-/M-oM-,M-^D) -> n--bssffl (expected xn--bssffl)
> 10012aef0
> 
> etc etc etc ...
> 
> I will say that every locale alive is installed here :
> 
> beta$
> beta$ locale -a | wc -l
>      345
> beta$
> 
> Generally I work with these in place :
> 
> beta$ env | sort | grep 'LC'
> LC_COLLATE=C
> LC_CTYPE=C
> LC_MESSAGES=C
> LC_MONETARY=C
> LC_NUMERIC=C
> LC_TIME=C
> beta$
> 
> Not sure what else to look at.
> 
>

signature.asc
Description: OpenPGP digital signature