coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

reading all data from FIFO with 'dd' without blocking


From: Assaf Gordon
Subject: reading all data from FIFO with 'dd' without blocking
Date: Tue, 18 Nov 2014 19:40:01 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0

Hello,

I'm looking for a way to read all available data from a FIFO without blocking, 
using coreutils (or other unix shell commands?).

The use-case:
1. A program (inotifywait) waits for a file-system event, then writes several 
lines to a pipe.
2. Another script waits for those events, and runs a program upon changes.

A simple method would be:
===
   SRC=/my/data/directory

   mkfifo myfifo
### InotifyWait will write several lines to "myfifo" on every file change in '$SRC'
   inotifywait --outfile myfifo --monitor "$SRC" &

   ### Wait for events, and run a program after each file change
   while read A ; do
      ## call external program
      my-program
   done < myfifo
===

I want to minimize the amount of times I run 'my-program'.
That is: if 'inotifywait' reported 10 changed files in ~20 lines to 'myfifo' 
(in one quick burst),
I want to group them together.

A better solution, using 'timeout' and 'cat' is:
===
   SRC=/my/data/directory

   mkfifo myfifo
### InotifyWait will write several lines to "myfifo" on every file change in '$SRC'
   inotifywait --outfile myfifo --monitor "$SRC" &

   ### Wait for events, and run a program after each file change
   while read A ; do
      ## Consume any events that happen in the next 1 second
      timeout 1 cat > /dev/null

      ## call external program
      my-program
   done < myfifo
===

But this results in a 1 second delay.

What I tried (but failed) to use is "dd iflag=nonblock":

===
   SRC=/my/data/directory

   mkfifo myfifo
### InotifyWait will write several lines to "myfifo" on every file change in '$SRC'
   inotifywait --outfile myfifo --monitor "$SRC" &

   ### Wait for events, and run a program after each file change
   ( while read A ; do
      ## Use DD to consume all available data from STDIN without blocking
      dd of=/dev/null iflag=nonblock

      ## call external program
      my-program
   done ) < myfifo
===

With the above script, 'dd' prints the following:
    dd: error reading ‘standard input’: Resource temporarily unavailable
    0+0 records in
    0+0 records out
    0 bytes (0 B) copied, 0.00012379 s, 0.0 kB/s

Is there a way to achieve this with 'dd' ?
Or with another program using the shell ?

(Note that for this application, I do not care about message-boundaries in the 
fifo - if a line gets truncated that's fine).

Thanks,
 - Assaf



reply via email to

[Prev in Thread] Current Thread [Next in Thread]