emacs-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[debbugs-tracker] bug#9157: closed ([PATCH] dd: sparse conv flag)


From: GNU bug Tracking System
Subject: [debbugs-tracker] bug#9157: closed ([PATCH] dd: sparse conv flag)
Date: Tue, 28 Feb 2012 21:51:02 +0000

Your message dated Tue, 28 Feb 2012 21:49:43 +0000
with message-id <address@hidden>
and subject line Re: bug#9157: [PATCH] dd: sparse conv flag
has caused the debbugs.gnu.org bug report #9157,
regarding [PATCH] dd: sparse conv flag
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
9157: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9157
GNU Bug Tracking System
Contact address@hidden with problems
--- Begin Message --- Subject: [PATCH] dd: sparse conv flag Date: Sat, 23 Jul 2011 20:08:01 +0400
---
 doc/coreutils.texi |    4 +++
 src/dd.c           |   22 +++++++++++++++-
 tests/Makefile.am  |    1 +
 tests/dd/sparse    |   70 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 96 insertions(+), 1 deletions(-)
 create mode 100755 tests/dd/sparse

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 424446c..761c698 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8127,6 +8127,10 @@ Pad every input block to size of @samp{ibs} with 
trailing zero bytes.
 When used with @samp{block} or @samp{unblock}, pad with spaces instead of
 zero bytes.
 
address@hidden sparse
address@hidden sparse
+Make sparse output file.
+
 @end table
 
 The following ``conversions'' are really file flags
diff --git a/src/dd.c b/src/dd.c
index 0824f6c..49847f5 100644
--- a/src/dd.c
+++ b/src/dd.c
@@ -126,7 +126,8 @@ enum
     C_NOCREAT = 010000,
     C_EXCL = 020000,
     C_FDATASYNC = 040000,
-    C_FSYNC = 0100000
+    C_FSYNC = 0100000,
+    C_SPARSE = 0200000
   };
 
 /* Status bit masks.  */
@@ -268,6 +269,7 @@ static struct symbol_value const conversions[] =
   {"sync", C_SYNC},            /* Pad input records to ibs with NULs. */
   {"fdatasync", C_FDATASYNC},  /* Synchronize output data before finishing.  */
   {"fsync", C_FSYNC},          /* Also synchronize output metadata.  */
+  {"sparse", C_SPARSE},                /* Make sparse output file. */
   {"", 0}
 };
 
@@ -533,6 +535,9 @@ Each CONV symbol may be:\n\
   fsync     likewise, but also write metadata\n\
 "), stdout);
       fputs (_("\
+  sparse    make sparse output file\n\
+"), stdout);
+      fputs (_("\
 \n\
 Each FLAG symbol may be:\n\
 \n\
@@ -985,6 +990,21 @@ iwrite (int fd, char const *buf, size_t size)
     {
       ssize_t nwritten;
       process_signals ();
+      if (conversions_mask & C_SPARSE)
+        {
+          off_t seek_size = 0;
+          while (buf[total_written + seek_size] == 0)
+            ++seek_size;
+          if (seek_size)
+            {
+              off_t cur_off = 0;
+              cur_off = lseek(fd, seek_size, SEEK_CUR);
+              if (cur_off < 0)
+                break;
+              total_written += seek_size;
+              continue;
+            }
+        }
       nwritten = write (fd, buf + total_written, size - total_written);
       if (nwritten < 0)
         {
diff --git a/tests/Makefile.am b/tests/Makefile.am
index ebd1b11..0f1376a 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -364,6 +364,7 @@ TESTS =                                             \
   dd/skip-seek                                 \
   dd/skip-seek2                                        \
   dd/skip-seek-past-file                       \
+  dd/sparse                                     \
   dd/stderr                                    \
   dd/unblock                                   \
   dd/unblock-sync                              \
diff --git a/tests/dd/sparse b/tests/dd/sparse
new file mode 100755
index 0000000..f0e0806
--- /dev/null
+++ b/tests/dd/sparse
@@ -0,0 +1,70 @@
+#!/bin/sh
+# Ensure that dd conv=sparse works.
+
+# Copyright (C) 2003, 2005-2011 Free Software Foundation, Inc.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ../src
+print_ver_ dd
+
+# sometimes we may read less than 1M
+dd if=/dev/zero of=sample0 count=1 bs=1M 2> /dev/null || fail=1
+[ "`stat -c %s sample0`" = "1048576" ] || fail=1
+dd if=/dev/urandom of=sample1 count=1 bs=1M 2> /dev/null || fail=1
+[ "`stat -c %s sample1`" = "1048576" ] || fail=1
+
+# test 1
+dd if=sample1 of=test1 seek=0 bs=1M 2> /dev/null || fail=1
+dd if=sample0 of=test1 seek=1 bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=test1 seek=2 bs=1M 2> /dev/null || fail=1
+
+# test 1-1
+dd if=test1 of=out1-1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out1-1-check bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out1-1-check seek=2 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out1-1`" = "`stat -c '%s %b %B' out1-1-check`" ] || 
fail=1
+
+# test 1-2
+dd if=test1 of=out1-2 seek=1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out1-2-check seek=1 bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out1-2-check seek=3 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out1-2`" = "`stat -c '%s %b %B' out1-2-check`" ] || 
fail=1
+
+# test 1-3
+dd if=test1 of=out1-3 seek=1 skip=1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out1-3-check seek=2 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out1-3`" = "`stat -c '%s %b %B' out1-3-check`" ] || 
fail=1
+
+# test 2
+dd if=sample0 of=test2 seek=0 bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=test2 seek=1 bs=1M 2> /dev/null || fail=1
+dd if=sample0 of=test2 seek=2 bs=1M 2> /dev/null || fail=1
+
+# test 2-1
+dd if=test2 of=out2-1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out2-1-check seek=1 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out2-1`" = "`stat -c '%s %b %B' out2-1-check`" ] || 
fail=1
+
+# test 2-2
+dd if=test2 of=out2-2 seek=1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out2-2-check seek=2 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out2-2`" = "`stat -c '%s %b %B' out2-2-check`" ] || 
fail=1
+
+# test 2-3
+dd if=test2 of=out2-3 seek=1 skip=1 conv=sparse bs=1M 2> /dev/null || fail=1
+dd if=sample1 of=out2-3-check seek=1 bs=1M 2> /dev/null || fail=1
+[ "`stat -c '%s %b %B' out2-3`" = "`stat -c '%s %b %B' out2-3-check`" ] || 
fail=1
+
+Exit $fail
-- 
1.7.0.4





--- End Message ---
--- Begin Message --- Subject: Re: bug#9157: [PATCH] dd: sparse conv flag Date: Tue, 28 Feb 2012 21:49:43 +0000 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110816 Thunderbird/6.0
On 02/28/2012 06:24 PM, Paul Eggert wrote:
> On 02/28/2012 09:01 AM, Pádraig Brady wrote:
> 
>> Doing that would be "dangerous" as I said above,
>> or in other words, surprising to users to not update
>> possibly non NUL data in the output file.
> 
> It's not surprising at all.  It's what I want and
> expect.  For example, suppose every nonzero byte of file A
> has an offset that is in a hole of file B.  Then I should
> be able to overlay A and B into another file C, as follows:
> 
> dd if=A of=C conv=sparse
> dd if=B of=C conv=sparse,notrunc
> 
> This is akin to the "or" operation on files, and it'd be a nice
> operation to have.  Even if A's contents don't fit in B's holes,
> I still might want to do the above, to allow B's non-holes to
> override A's contents.  Why disable this useful functionality?
> 
> Any surprise issues can be dealt with by documenting dd's
> behavior appropriately.

Fair enough. So this is the "write mask" functionality
I referred to, and which is now supported consistently
since the write(fd,"\0",1) to ftruncate() change.

>> +Try to seek rather than write @sc{nul} output blocks.
>> +This will create sparse output when extending.
>> +This option is ignored in conjunction with
>> address@hidden or @samp{oflag=append}.
> 
> I still dubious about this level of handholding.
> dd is meant for low-level use, and as far as possible
> options should be orthogonal.  For example, with dd,
> oflag=append does not disable seek=N -- both flags operate,
> which means that the seek is ineffective unless it
> is past the end of file.  conv=sparse oflag=append
> should be similar: all it should mean is that the
> writes are sparse when they're past the end of
> the file (this latter functionality should work, but
> doesn't work with the proposed patch).

OK I'll remove the 2 restrictions, and essentially
move the comments for them from code to texinfo.

cheers,
Pádraig.


--- End Message ---

reply via email to

[Prev in Thread] Current Thread [Next in Thread]