gawk 3.1.2g problems with NaNs and substr etc.

[Paul Eggert]

gawk test version 3.1.2g has a few problems when applying weird
numeric values like NaN to operations like substr that expect
integers.  Here are a few examples illustrating the problem.

$ ./gawk-3.1.2g -W lint 'BEGIN {print substr("x", 1e300*1e300-1e300*1e300)}' 
</dev/null
gawk-3.1.2g: cmd. line:1: warning: substr: non-integer start index NaN will be 
truncated
gawk-3.1.2g: cmd. line:1: warning: substr: start index NaN is past end of string

$ ./gawk-3.1.2g -W lint 'BEGIN {print substr("x", 1, 0.1)}' </dev/null
gawk-3.1.2g: cmd. line:1: warning: substr: non-integer length 0.1 will be 
truncated

$ ./gawk-3.1.2g -W lint 'BEGIN {print substr("x", 1, 1e300*1e300-1e300*1e300)}' 
</dev/null
gawk-3.1.2g: cmd. line:1: warning: substr: non-integer length NaN will be 
truncated
gawk-3.1.2g: cmd. line:1: warning: substr: length NaN at start index 1 exceeds 
length of first argument (1)
x


I looked for examples of all the problems that I could find, and came
up with the following patch.  I probably missed some problems, but
this should be better than nothing.  NaN is a bit tricky, since it
never compares equal to, less than, or greater than anything
(including itself).

2003-07-06  Paul Eggert  <address@hidden>

        * builtin.c (do_substr): Issue better diagnostics when
        d_substr and d_length are NaN, or when 0 < d_length < 1.
        Be careful when comparing double to SIZE_MAX, as
        the comparison might return the "wrong" answer when
        (double) SIZE_MAX is a number that is not equal to
        SIZE_MAX.
        (do_gensub): Watch out for HOW values that are out of range
        or are NaN.
        (do_dcngettext): dcngettext wants an argument of type
        unsigned long, not long, so use a value of that type.

--- builtin.c   2003/06/15 16:52:32     3.1
+++ builtin.c   2003/07/06 06:54:49
@@ -1232,7 +1232,7 @@ do_substr(NODE *tree)
        d_index = force_number(t2);
        free_temp(t2);
 
-       if (d_index < 1.0) {
+       if (! (d_index >= 1)) {
                if (do_lint)
                        lintwarn(_("substr: start index %g is invalid, using 
1"),
                                 d_index);
@@ -1256,11 +1256,11 @@ do_substr(NODE *tree)
                t3 = tree_eval(tree->rnode->rnode->lnode);
                d_length = force_number(t3);
                free_temp(t3);
-               if (d_length <= 0.0) {
+               if (! (d_length >= 1)) {
                        if (do_lint == LINT_ALL)
-                               lintwarn(_("substr: length %g is <= 0"), 
d_length);
-                       else if (do_lint == LINT_INVALID && d_length < 0)
-                               lintwarn(_("substr: length %g is < 0"), 
d_length);
+                               lintwarn(_("substr: length %g is not >= 1"), 
d_length);
+                       else if (do_lint == LINT_INVALID && ! (d_length >= 0))
+                               lintwarn(_("substr: length %g is not >= 0"), 
d_length);
                        free_temp(t1);
                        return Nnull_string;
                }
@@ -1275,7 +1275,7 @@ do_substr(NODE *tree)
                        _("substr: length %g too big for string indexing, 
truncating to %g"),
                                        d_length, (double) SIZE_MAX);
                }
-               if (d_length <= SIZE_MAX)
+               if (d_length < SIZE_MAX)
                        length = d_length;
                else
                        length = SIZE_MAX;
@@ -2291,10 +2291,12 @@ do_gensub(NODE *tree)
                        how_many = 1;
        } else {
                d = force_number(t);
-               if (d > 0)
+               if (d < 1)
+                       how_many = 1;
+               else if (d < LONG_MAX)
                        how_many = d;
                else
-                       how_many = 1;
+                       how_many = LONG_MAX;
                if (d == 0)
                        warning(_("gensub: 3rd argument of 0 treated as 1"));
        }
@@ -2825,7 +2827,7 @@ do_dcngettext(NODE *tree)
 {
        NODE *tmp, *t1, *t2, *t3;
        char *string1, *string2;
-       long number;
+       unsigned long number;
        char *the_result;
 #if ENABLE_NLS && HAVE_LC_MESSAGES && HAVE_DCGETTEXT
        int lc_cat;
@@ -2841,7 +2843,7 @@ do_dcngettext(NODE *tree)
        string2 = t2->stptr;
 
        tmp = tree->rnode->rnode->lnode;        /* third argument */
-       number = (long) double_to_int(force_number(tree_eval(tmp)));
+       number = (unsigned long) double_to_int(force_number(tree_eval(tmp)));
 
        t3 = NULL;
 #if ENABLE_NLS && HAVE_LC_MESSAGES && HAVE_DCGETTEXT