Re: [PATCH v3 0/4] Use AF_ALG in checksum utilities

[Bruno Haible]

The 4 tests that I added all fail on the empty file. Example:

$ ./test-sha1 
sha1_stream produced wrong result.
Expected: 
\xda\x39\xa3\xee\x5e\x6b\x4b\x0d\x32\x55\xbf\xef\x95\x60\x18\x90\xaf\xd8\x07\x09
Got:      
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

Seen on Ubuntu 16.04 LTS with a kernel version 4.4.0-119, which somewhat
corresponds to the kernel.org version 4.4.114 [1].

[1] https://people.canonical.com/~kernel/info/kernel-version-map.html

This workaround makes the tests pass:


2018-05-05  Bruno Haible  <address@hidden>

        af_alg: Fix bug on empty files.
        * lib/af_alg.c (afalg_stream): Ignore the kernel's result if the input
        stream is empty.

diff --git a/lib/af_alg.c b/lib/af_alg.c
index 1f4a6aa..06344b1 100644
--- a/lib/af_alg.c
+++ b/lib/af_alg.c
@@ -83,10 +83,12 @@ afalg_stream (FILE * stream, const char *alg, void 
*resblock, ssize_t hashlen)
   else
     {
       /* sendfile not possible, do a classic read-write loop.  */
+      int non_empty = 0;
       ssize_t size;
       char buf[BLOCKSIZE];
       while ((size = fread (buf, 1, sizeof buf, stream)))
         {
+          non_empty = 1;
           if (send (ofd, buf, size, MSG_MORE) != size)
             {
               ret = -EIO;
@@ -98,6 +100,13 @@ afalg_stream (FILE * stream, const char *alg, void 
*resblock, ssize_t hashlen)
           ret = -EIO;
           goto out_ofd;
         }
+      /* On Linux 4.4.0 at least, the value for an empty stream is wrong
+         (all zeroes).  */
+      if (!non_empty)
+        {
+          ret = -EAFNOSUPPORT;
+          goto out_ofd;
+        }
     }
 
   if (read (ofd, resblock, hashlen) != hashlen)