Re: piping tail -f through cut

[Bob Proulx]

Bob van Loosen wrote:
> Whenever I do this:
> 
> tail -f  /var/log/kern.log | cut -f 8- -d " " | dd bs=1
> 
> I get no output, but when I do this:
> 
> tail -f  /var/log/kern.log | cut -f 8- -d " "
> 
> I do get output, I'm using coreutils 5.97 on Ubuntu 7.10 64 bit.

What you are seeing are the effects of buffering on output I/O.  The C
standard I/O library used by programs to write output data buffers
output into large chunks in order to run fast and efficiently.  This
means that data to be written by a program is saved up into a buffer
until the buffer is full.  When enough output data is available such
that a full buffer can be output all at once then the data is
written.  When there is less than a full buffer block of data it is
held in memory waiting for more to arrive.

The C standard I/O library determines if it should buffer output based
upon whether the output is a file or a terminal.  In your first case
'cut | dd' the cut program's stdout is connected to a pipe.  Because
of this the library enables buffering.  In your second case the cut
program's stdout is connected to your terminal.  Because the output is
to a terminal the C library assumes that a person reading it wouldn't
want to wait and disables buffering.

The kern.log on your system is not very active.  It is not generating
very much data.  The 'tail -f' is therefore not writing very much data
to cut.  The cut program is not writing very much data to dd.  Because
there is insufficient data to fill a buffer block the data is holding
off.  If the file were more active then the buffers would fill up and
the data would flow.  But since there isn't enough data that isn't
happening.

The 'tail -f' program also never closes its output.  It is expected to
run until interrupted.  Because tail's output is not closed it means
that cut's input is not closed.  Normally when the input is closed a
program will exit which closes its output.  This cascades through the
pipeline and every program will flush its output buffers and exit and
all data is processed.  But because 'tail -f' never closes its output
the cut program is always waiting for more input.  Because the cut
program never closes its output the dd program is always waiting for
more input.

The use of 'bs=1' doesn't help because the cut program is the one that
is buffering the output.

Hope this helps,

Bob