Re: [bug-gawk] Problem with printing 5000 lines to a coprocess

[Andrew J. Schorr]

On Sat, Dec 20, 2014 at 09:10:48PM -0200, Hermann Peifer wrote:
> However, while re-reading the manual, I saw this hint at the end of section
> 12.3:
> 
> > You may also use pseudo-ttys (ptys) for two-way communication
> > instead of pipes (...) Using ptys avoids the buffer deadlock
> > issues described earlier, at some loss in performance.
> 
> As far as I can tell: my initial approach (write 1 line/read 1 line) works
> fine now, after adding: PROCINFO[command, "pty"] = 1

Hmmm.  The "pty" trick is just a way to solve the flushing problem.  The stdio
library tries to detect whether stdout and stderr are going to a terminal.  If
yes, then stdio uses line buffering for stdout and stderr; i.e. it flushes
output after every newline feed.  When stdout & stderr are not to a terminal,
stdio will buffer output in fixed blocks of something like 4096 bytes.  The
reason to use a pty for a coprocess is to trick the child process into thinking
that it's writing to a terminal so that stdio will use line-buffering without
being explicitly instructed to do so.  If you write a program that you intend
to use as a coprocess, you should have it call 'setlinebuf(stdout)' and/or
'setlinebuf(stderr)'.  I usually add a command-line option to my programs
to request line-buffering.

In other words, I don't think using a pty solves your conceptual problem.
In practice, I suspect that using the pty is effectively increasing the 
size of the kernel buffers between the chlid process and gawk.  Perhaps
it is doubling the amount of buffer space.  If you increase the problem size,
you should find that your code will hang again.

> As you mentioned earlier: maybe some additional hints about buffer deadlocks
> could be added to the manual.

This is probably a good idea.

Maybe we also need a gawk extension library to provide access to the "mkstemp"
library call.  That way, it would become easy to create a unique temporary
file.

Regards,
Andy