Re: seekable stdin test failure on OS X

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Gary V. Vaughan on 4/2/2007 6:44 AM:
> Hi Eric,

Hi Gary, and adding gnulib readers,

>>
>> Yuck.  I think this means that on your platform, fflush(stdin) is
>> neglecting to reset the underlying fd position when stdin is seekable,
>> which is a bug in your libc.  To confirm, do you have anything like
>> strace
>> or truss that you could run to see what syscalls took place?
> 
> Yes, there is ktrace which outputs the following when running test 68:

> The fork and wait must be the syscmd, and the lseek(0,0,0x1) that returns
> 24 is the seek on stdin from fflush.  What does that mean though?

Not quite - read on.

> 
>   6760 m4       GIO   fd 0 read 24 bytes
>        "syscmd(`cat')m4exit(15)
>        "
>   6760 m4       RET   read 24/0x18
>   6760 m4       CALL  lseek(0,0,0x1)
>   6760 m4       RET   lseek 24/0x18

OK, this occurs in modules/m4.c, in m4_sysval_flush.  The lseek(0,0,1) is
m4's attempt to learn whether stdin is seekable (and it is, since the
result is 24), prior to calling fflush (since POSIX only requires the
repositioning on seekable files).  But then the fflush/fseek pair makes no
syscalls, which is wrong, since POSIX requires that fflush followed
immediately by fseek discard all unprocessed data on input streams (here,
"m4exit(15)\n" is unprocessed data) and set the position of the underlying
fd.  I would submit a bug to the MacOS people.  In the case of cygwin,
this shows up as another call to lseek, putting the offset back to 13.

>   6760 m4       CALL  sigaction(0x2,0xbfffece0,0xbfffed3c)
...
>   6760 m4       RET   sigprocmask 0

Yes, this sequence was the syscmd.  And cat should have seen the text
"m4exit(15)\n", seeing as how the fflush was supposed to reset the
underlying fd location back to the unprocessed data,...

>   6760 m4       CALL  fstat(0x1,0xbfffecc0)
>   6760 m4       RET   fstat 0
>   6760 m4       CALL  fstat(0x2,0xbfffec60)
>   6760 m4       RET   fstat 0
>   6760 m4       CALL  lseek(0,0,0x1)
>   6760 m4       RET   lseek 24/0x18

This lseek was correct - after invoking a child process, the fd's
underlying position may have been changed by the child, so libc must
indeed rediscover where to resume reading from.  In this case, since cat
should have consumed the rest of the input file, there should be nothing
else available for reading from stdin, and m4 should exit with status 0
for normal end of file processing, rather than 15 due to reparsing data
consumed by the child process.

> 
>> We may end up having to write an autoconf test that can detect this
>> brokenness in libc, and based on that test, add additional calls to fseek
>> if using fflush alone is inadequate for m4 to follow the POSIX rules of
>> how to handle seekable input streams.
> 
> Bleh!

I agree.  But will it even work?  m4 already called fseek, but currently
it just discarded the results; can you step through m4_sysval_flush to
determine if the fseek was accurately returning the offset in the middle
of the input rather than at the end?  If it does work, how many other
gnulib-using programs would benefit from a fix?

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGEQQl84KuGfSFAYARAsCOAKDOG3+VmSX3B6uuWEXT2KLRjmbcmQCbB0Mp
Hm/U+VRAJAvhDvOgtdodwlU=
=CJBD
-----END PGP SIGNATURE-----