Re: bignum branch

[Andy Moreton]

On Mon 16 Jul 2018, Andy Moreton wrote:

> On Mon 16 Jul 2018, Tom Tromey wrote:
>
>>>>>>> "Tom" == Tom Tromey <address@hidden> writes:
>>
>> Tom> I was thinking this is what I' have emacs do when
>> Tom> sizeof(EMACS_INT) > sizeof(long).
>>
>> Please try this patch.
>>
>> Unfortunately I don't know how I can test it locally
>
> This builds ok on Windows with 64bit mingw64 (MSYS2), but still has a
> few issues and some odd behaviour:
>
> 3) A problem with the ccl.el compiler for CCL programs. I think that
>    this can be fixed by truncating the CCL code words output by the
>    compiler in ccl.el to ensure that they doe not extend beyond the
>    expected 28bit code word. The interpreter in ccl.c appears to range
>    check the code words properly.
>
> In toplevel form:
> ../../../lisp/obsolete/pgg.el:29:1:Error: Error in CCL program
> In toplevel form:
> ../../../lisp/obsolete/pgg-gpg.el:32:1:Error: Error in CCL program
> In toplevel form:
> ../../../lisp/obsolete/pgg-pgp.el:30:1:Error: Error in CCL program
> In toplevel form:
> ../../../lisp/obsolete/pgg-pgp5.el:30:1:Error: Error in CCL program

I think the following patch fixes the issues in CCL (compiler,
interpreter and disassembler). It truncates the compiled CCL code words
to 28 bits and then sign extends to ensure that the interpreter sees
signed values of constants (stored at the upper end of the code word).

I've done some simple testing with:

    ;; check compiler output
    (require 'pgg)
    pgg-parse-crc24
    (seq-map (lambda (x) (format "#x%x" x)) pgg-parse-crc24)

    ;; check disassembler
    (ccl-dump pgg-parse-crc24)

    ;; check interpreter
    (dolist (s '("foo" "bar" "baz"))
      (let ((crc (pgg-parse-crc24-string s)))
        (insert "\n" s " --> "
                (mapconcat (lambda (b) (format "%02x" b)) crc ""))))

The patch has been tested this on Windows 32bit and 64bit emacs builds
(mingw64 MSYS2) on master and the feature/bignum branch.

    AndyM

Fix CCL compiler and interpreter reliance on fixnum behaviour

        * lisp/international/ccl.el (ccl-embed-data)
        (ccl-embed-current-address): Truncate code word to 28 bits to
        ensure fixnum values in CCL compiler output.
        (ccl-get-next-code): Truncate code word to 28 bits and sign
        extend to fix output from ccl-dump.
        * src/ccl.c (GET_CCL_RANGE): Truncate code word to 28 bits and
        sign extend to ensure fixnum values in CCL interpreter.

diff --git a/lisp/international/ccl.el b/lisp/international/ccl.el
index d2f490d59c..e08d9fc55e 100644
--- a/lisp/international/ccl.el
+++ b/lisp/international/ccl.el
@@ -187,6 +187,7 @@ ccl-current-ic
 (defun ccl-embed-data (data &optional ic)
   "Embed integer DATA in `ccl-program-vector' at `ccl-current-ic' and
 increment it.  If IC is specified, embed DATA at IC."
+  (setq data (logand #x0fffffff data))
   (if ic
       (aset ccl-program-vector ic data)
     (let ((len (length ccl-program-vector)))
@@ -230,7 +231,8 @@ ccl-embed-current-address
 `ccl-program-vector' at IC without altering the other bit field."
   (let ((relative (- ccl-current-ic (1+ ic))))
     (aset ccl-program-vector ic
-         (logior (aref ccl-program-vector ic) (ash relative 8)))))
+         (logior (aref ccl-program-vector ic)
+                  (logand #x0fffffff (ash relative 8))))))
 
 (defun ccl-embed-code (op reg data &optional reg2)
   "Embed CCL code for the operation OP and arguments REG and DATA in
@@ -986,7 +988,9 @@ ccl-dump
 (defun ccl-get-next-code ()
   "Return a CCL code in `ccl-code' at `ccl-current-ic'."
   (prog1
-      (aref ccl-code ccl-current-ic)
+      (let ((code (aref ccl-code ccl-current-ic)))
+        ;; Truncate to 28 bit code word and sign extend
+        (- (logxor (logand #xfffffff code) #x8000000) #x8000000))
     (setq ccl-current-ic (1+ ccl-current-ic))))
 
 (defun ccl-dump-1 ()
diff --git a/src/ccl.c b/src/ccl.c
index 529b302ed9..80e9bcd98a 100644
--- a/src/ccl.c
+++ b/src/ccl.c
@@ -737,6 +737,9 @@ while (0)
   do                                                           \
     {                                                          \
       EMACS_INT prog_word = XINT ((ccl_prog)[ic]);             \
+      /* Truncate to 28 bit code word and sign extend */       \
+      prog_word = prog_word & 0x0fffffff;                      \
+      prog_word = (prog_word ^ 0x08000000) - 0x08000000;       \
       if (! ASCENDING_ORDER (lo, prog_word, hi))               \
        CCL_INVALID_CMD;                                        \
       (var) = prog_word;                                       \