[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: sed on binary files

From: Eric Blake
Subject: Re: sed on binary files
Date: Thu, 02 Oct 2008 06:16:16 -0600
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/20080914 Thunderbird/ Mnenhy/

Hash: SHA1

According to Gary V. Vaughan on 10/1/2008 10:09 PM:
>> I'm working on making m4 1.6 transparently handle NUL,
> Excellent!  I made an attempt to do that myself on the 2.0 branch some
> years ago, but it didn't go well so I never committed...

The argv_ref branch already does this; it is just a matter of finishing
porting it to branch-1.6 and master (and in the process of that porting, I
discovered that the tests I wrote worked fine under GNU sed but died under
Solaris 10).

>> or can have false positives (if both
>> stderr and expected error are normalized, then regressions involving
>> added
>> or missing NUL are not detected).  I don't want to require perl for just
>> this one test; m4 seems fundamental enough to keep the testsuite
>> restricted to the GNU coding standards set of tools.
> I'd be inclined to do that in C.  A few lines should be sufficient to
> write a minimal filter that writes '\' '0' or '^' '@' to output whenever
> a NUL byte arrives?

Actually, I'm a bit lazy - I guess I'm okay with false positives on
Solaris when using deficient sed, so long as we can also run on Solaris
with GNU sed.  So I'm installing this patch, which lets the user select
the right sed, as well as passing both files through sed (a no-op for GNU
sed, but strips NUL bytes equally for Solaris sed).  (At any rate, it was
easier to code than searching for a tr that handles NUL).

Should I also modify configure.ac to call AC_PROG_SED, and feed that as
the default for $SED in the check script?  (The master branch is currently
the only branch that uses $SED, thanks to libtool.)

- --
Don't work too hard, make some time for fun as well!

Eric Blake             address@hidden
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

>From dc967b18e45bed9591a508ed16bbf4cfa3b0e6b9 Mon Sep 17 00:00:00 2001
From: Eric Blake <address@hidden>
Date: Thu, 2 Oct 2008 06:11:51 -0600
Subject: [PATCH] Work around Solaris' sed inability to process NUL.

* checks/check-them (SED): Allow user to override.
(err): Postprocess with sed, just like xerr, to avoid differences
on platforms where sed strips NUL.

Signed-off-by: Eric Blake <address@hidden>
 ChangeLog         |    7 +++++++
 checks/check-them |   33 ++++++++++++++++++++-------------
 2 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 31219e4..780b635 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2008-10-02  Eric Blake  <address@hidden>
+       Work around Solaris' sed inability to process NUL.
+       * checks/check-them (SED): Allow user to override.
+       (err): Postprocess with sed, just like xerr, to avoid differences
+       on platforms where sed strips NUL.
 2008-09-25  Eric Blake  <address@hidden>
        Tweak error message on command line failure.
diff --git a/checks/check-them b/checks/check-them
index 89f8b41..017e13b 100755
--- a/checks/check-them
+++ b/checks/check-them
@@ -41,6 +41,9 @@ skipped=
+# Allow user to select sed
+: ${SED=sed}
 # Find out where the examples live.
 if test "x$1" = x-I ; then
@@ -56,7 +59,7 @@ if test "x$1" = x-m ; then
 # Find out how the executable prints argv[0]
-m4name=`"$m4" --help | sed -e 's/Usage: \(.*\) \[OPTION.*/\1/' \
+m4name=`"$m4" --help | ${SED} -e 's/Usage: \(.*\) \[OPTION.*/\1/' \
   -e 's/\\\\/\\\\\\\\/g' -e 1q`
 # Find out if we should strip \r in the output
@@ -70,7 +73,7 @@ else
 # Find out what version the executable claims to be
-m4version=`sed -n 's/.* //p;q' $out`
+m4version=`${SED} -n 's/.* //p;q' $out`
 # Find out if diff supports useful options.
 if diff -u /dev/null /dev/null 2>/dev/null ; then
@@ -88,12 +91,12 @@ do
   echo "Checking $file"
-  options=`sed -ne '3s/^dnl @ extra options: //p;3q' "$file"`
-  sed -e '/^dnl @/d' -e '/^\^D$/q' "$file" \
+  options=`${SED} -ne '3s/^dnl @ extra options: //p;3q' "$file"`
+  ${SED} -e '/^dnl @/d' -e '/^\^D$/q' "$file" \
     | LC_MESSAGES=C M4PATH=$examples "$m4" -d $options - >$out 2>$err
-  xstat=`sed -ne '2s/^dnl @ expected status: //p;2q' "$file"`
+  xstat=`${SED} -ne '2s/^dnl @ expected status: //p;2q' "$file"`
   case $stat in
       skipped="$skipped $file"
@@ -103,29 +106,33 @@ do
     $xstat) ;;
       failed="$failed $file:status"
-      echo `sed -e 's/^dnl //' -e 1q $file`
+      echo `${SED} -e 's/^dnl //' -e 1q $file`
       echo "$file: status was $stat, expected $xstat"
-  xoutfile=`sed -n 's/^dnl @ expected output: //p' "$file"`
+  xoutfile=`${SED} -n 's/^dnl @ expected output: //p' "$file"`
   if test -z "$xoutfile" ; then
-    sed -e '/^dnl @result{}/!d' -e 's///' -e "s|examples/|$examples/|" \
+    ${SED} -e '/^dnl @result{}/!d' -e 's///' -e "s|examples/|$examples/|" \
       -e "s|@value{VERSION}|$m4version|" "$file" > $xout
     cp "$examples/$xoutfile" $xout
-  xerrfile=`sed -n 's/^dnl @ expected error: //p' "$file"`
+  xerrfile=`${SED} -n 's/^dnl @ expected error: //p' "$file"`
   case $xerrfile in
       cp $err $xerr ;;
-      sed '/^dnl @error{}/!d
+      ${SED} '/^dnl @error{}/!d
           s///; '"s|^m4:|$m4name:|; s|examples/|$examples/|" \
         "$file" > $xerr ;;
-      sed "s|^m4:|$m4name:|; s|examples/|$examples/|" \
+      # This is a no-op for GNU sed, but is important for Solaris sed
+      # which strips NUL bytes from input.
+      ${SED} 's,x,x,' $err > $err.1
+      mv $err.1 $err
+      ${SED} "s|^m4:|$m4name:|; s|examples/|$examples/|" \
         "$examples/$xerrfile" > $xerr ;;
@@ -145,7 +152,7 @@ do
     failed="$failed $file:out"
-    echo `sed -e 's/^dnl //' -e 1q $file`
+    echo `${SED} -e 's/^dnl //' -e 1q $file`
     echo "$file: stdout mismatch"
     diff $diffopts $xout $out
@@ -154,7 +161,7 @@ do
     failed="$failed $file:err"
-    echo `sed -e 's/^dnl //' -e 1q $file`
+    echo `${SED} -e 's/^dnl //' -e 1q $file`
     echo "$file: stderr mismatch"
     diff $diffopts $xerr $err

reply via email to

[Prev in Thread] Current Thread [Next in Thread]