Re: [bug-gawk] $0 reassignment corruption in 4.2

[Andrew J. Schorr]

Hi,

On Fri, Nov 24, 2017 at 11:19:52AM +0200, Arnold Robbins wrote:
> Andy:
> > Arnold -- does this look right to you? I haven't chased down all the
> > nuances of what this assign function is doing...
> >
> > Note: we may have a similar ordering problem at debug.c:1301.
> 
> This seems to have been taken care of by the earlier change.
> I could not produce a problem here.  (It's a little weird running
> the gawk debugger from inside gdb... :-)
> 
> I think we're good.

I don't think so. I ran a debugger session under valgrind, and I see this:

bash-4.2$ ./gawk --version
GNU Awk 4.2.0, API: 2.0 (GNU MPFR 3.1.1, GNU MP 6.0.0)
Copyright (C) 1989, 1991-2017 Free Software Foundation.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
bash-4.2$ cat /tmp/test.awk
function reassign(x, y) {
   print "hello"
   # $0 = x
   print y
}

BEGIN {
   $0 = substr("geronimo", 5, 3)
   reassign(" 52", $1)
}
bash-4.2$ valgrind ./gawk -D -f /tmp/test.awk
==12073== Memcheck, a memory error detector
==12073== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==12073== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==12073== Command: ./gawk -D -f /tmp/test.awk
==12073== 
gawk> break reassign
==12073== Syscall param sendmsg(msg.msg_name) points to uninitialised byte(s)
==12073==    at 0x5B4E3B0: __sendmsg_nocancel (in /usr/lib64/libc-2.17.so)
==12073==    by 0x4E4C30E: readline (in /usr/lib64/libreadline.so.6.2)
==12073==    by 0x42F4C9: zzlex (command.y:1050)
==12073==    by 0x42F4C9: zzparse (command.c:1451)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Address 0xffeffee82 is on thread 1's stack
==12073==  in frame #1, created by readline (???:)
==12073== 
Breakpoint 1 set at file `/tmp/test.awk', line 2
gawk> run
Starting program: 
Stopping in BEGIN ...
Breakpoint 1, reassign(x, y) at `/tmp/test.awk':2
2          print "hello"
gawk> set $0 = "junk"
==12073== Syscall param sendmsg(msg.msg_name) points to uninitialised byte(s)
==12073==    at 0x5B4E3B0: __sendmsg_nocancel (in /usr/lib64/libc-2.17.so)
==12073==    by 0x4E4C30E: readline (in /usr/lib64/libreadline.so.6.2)
==12073==    by 0x42F4C9: zzlex (command.y:1050)
==12073==    by 0x42F4C9: zzparse (command.c:1451)
==12073==    by 0x436AEB: debug_pre_execute (debug.c:3676)
==12073==    by 0x44405A: h_interpret (interpret.h:88)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Address 0xffeffe512 is on thread 1's stack
==12073==  in frame #1, created by readline (???:)
==12073== 
==12073== Invalid read of size 2
==12073==    at 0x4C2E0D0: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1018)
==12073==    by 0x449E05: purge_record (field.c:353)
==12073==    by 0x437648: do_set_var (debug.c:1305)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x436AEB: debug_pre_execute (debug.c:3676)
==12073==    by 0x44405A: h_interpret (interpret.h:88)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Address 0x6085000 is 0 bytes inside a block of size 4 free'd
==12073==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==12073==    by 0x45F418: r_unref (node.c:480)
==12073==    by 0x4377CC: unref (awk.h:1887)
==12073==    by 0x4377CC: do_set_var (debug.c:1302)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x436AEB: debug_pre_execute (debug.c:3676)
==12073==    by 0x44405A: h_interpret (interpret.h:88)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Block was alloc'd at
==12073==    at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==12073==    by 0x460074: emalloc_real (awk.h:1950)
==12073==    by 0x460074: make_str_node (node.c:386)
==12073==    by 0x424210: do_substr (builtin.c:1869)
==12073==    by 0x4462A4: h_interpret (interpret.h:967)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073== 
==12073== Invalid read of size 1
==12073==    at 0x4C2E240: memcpy@@GLIBC_2.14 (vg_replace_strmem.c:1018)
==12073==    by 0x449E05: purge_record (field.c:353)
==12073==    by 0x437648: do_set_var (debug.c:1305)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x436AEB: debug_pre_execute (debug.c:3676)
==12073==    by 0x44405A: h_interpret (interpret.h:88)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Address 0x6085002 is 2 bytes inside a block of size 4 free'd
==12073==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==12073==    by 0x45F418: r_unref (node.c:480)
==12073==    by 0x4377CC: unref (awk.h:1887)
==12073==    by 0x4377CC: do_set_var (debug.c:1302)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x436AEB: debug_pre_execute (debug.c:3676)
==12073==    by 0x44405A: h_interpret (interpret.h:88)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073==  Block was alloc'd at
==12073==    at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==12073==    by 0x460074: emalloc_real (awk.h:1950)
==12073==    by 0x460074: make_str_node (node.c:386)
==12073==    by 0x424210: do_substr (builtin.c:1869)
==12073==    by 0x4462A4: h_interpret (interpret.h:967)
==12073==    by 0x43BB94: do_run (debug.c:2960)
==12073==    by 0x42FDCB: zzparse (command.y:172)
==12073==    by 0x43BEA1: debug_prog (debug.c:2835)
==12073==    by 0x40DB22: main (main.c:501)
==12073== 
$0 = "junk"

With my proposed patch, this problem goes away.

> > And at interpret.h:373, I'm not sure how Op_field_spec_lhs really works
> > and what's going on with the field_assign value...
> > Still a mystery to me.
> 
> The way to understand when an opcode is used is to see in awkgram.y
> where it's generated.  The Op_xxx_lhs opcodes are used in cases
> where a field, variable, or array element need to be able to be
> assigned to. It's used for getline and also sub/gsub.
> 
> I didn't try to test anything but from the code, I think we're OK.

I'll defer to you on that one. It's a tangled web of code.

Regards,
Andy