dd skip= misbehavior

[Solar Designer]

Hello,

I am not sure whether this is a bug or not, but it's behavior I did not
expect:

In coreutils 5.94, as well as in older versions, dd.c: skip() does:

/* Throw away RECORDS blocks of BLOCKSIZE bytes on file descriptor FDESC,
   which is open with read permission for FILE.  Store up to BLOCKSIZE
   bytes of the data at a time in BUF, if necessary.  RECORDS must be
   nonzero.  If fdesc is STDIN_FILENO, advance the input offset.
   Return the number of records remaining, i.e., that were not skipped
   because EOF was reached.  */

static uintmax_t
skip (int fdesc, char const *file, uintmax_t records, size_t blocksize,
      char *buf)
[...]
      do
        {
          ssize_t nread = iread (fdesc, buf, blocksize);
[...]
          if (fdesc == STDIN_FILENO)
            advance_input_offset (nread);
        }
      while (--records != 0);

      return records;

Now, what happens when nread is smaller than blocksize?  The input
offset is advanced properly, however only the requested number of
_partial_ records is skipped before dd starts to actually copy data from
its input to its output.

The particular command I was using was:

ssh address@hidden 'cat hda2.gz' | gzip -d | dd bs=1024k of=/dev/hda2 
skip=33000 seek=33000

I wanted to skip the first 33,000 MB, then restore the remainder of the
partition from the backup copy.  However, gzip was making the data
available in 32 KB blocks, so only 33,000 of 32 KB blocks - not of 1 MB
blocks - were skipped on input, and the wrong data got written at
33,000 MB into the target partition.  The command ended with:

dd: writing `/dev/hda2': No space left on device

although the size of compressed data in hda2.gz exactly matched the
target partition size - that's another effect of the same problem.

POSIX.1-2001 does not appear to explicitly specify the behavior of
skip= with partial reads.  It defines skip= as follows:

     skip=n
             Skip n input blocks (using the specified input block size)
             before starting to copy. On seekable files, the implementation
             shall read the blocks or seek past them; on non-seekable files,
             the blocks shall be read and the data shall be discarded.

However, it gives this example:

     The following command:

 dd ibs=10 skip=1

     strips the first 10 bytes from standard input.

Notice that they said "the first 10 bytes", with no exceptions - does
this mean that this command should skip the first 10 bytes even if the
first read() returns only 5 bytes?

-- 
Alexander Peslyak <solar at openwall.com>
GPG key ID: B35D3598  fp: 6429 0D7E F130 C13E C929  6447 73C3 A290 B35D 3598
http://www.openwall.com - bringing security into open computing environments