[Chicken-janitors] #1572: read/write invariance of #! is violated

[Chicken Trac]

#1572: read/write invariance of #! is violated
----------------------------+--------------------------------
 Reporter:  zbigniew        |                 Owner:
     Type:  defect          |                Status:  new
 Priority:  major           |             Milestone:  someday
Component:  core libraries  |               Version:  5.0.0
 Keywords:                  |  Estimated difficulty:  easy
----------------------------+--------------------------------
 In Chicken 5, read/write invariance of symbols beginning with #! is
 violated.

 {{{
 > (with-input-from-string
     (with-output-to-string (lambda () (write (string->symbol "#!abc"))))
 read)
 Error: invalid `#!' token: “abc"

 > (with-input-from-string
     (with-output-to-string (lambda () (write (string->symbol "#!"))))
 read)
 Error: invalid `#!' token: “"
 }}}

 This causes a problem with chicken-doc, as there is a syntax entry called
 “#!” which is written out but cannot be read back in. In Chicken 4, this
 symbol is escaped when written out and reads back in fine.

 This occurs because in Chicken 5, sym-is-readable? was changed:

  commit 05f341e07a179ea95d891e5c55b2fe3d0dbe1ffe
  Author: Evan Hanson <address@hidden>
  Date:   Wed Mar 14 17:53:00 2018 +1300

  Print #!-style symbols verbatim, without pipes

  These symbols are readable, so should be printed as-is by `##sys#print'
 just like #:keywords or the #!eof token.

 {{{
  -                       ((and (eq? c #\#)
  -                            (not (eq? #\% (##core#inline "C_subchar" str
 1))))
  -                       #f)
  +                      ((eq? c #\#) ;; #!rest, #!key etc
  +                       (eq? (##core#inline "C_subchar" str 1) #\!))
 }}}


 Unfortunately this assertion is incorrect: not all #! style symbols are
 readable, only valid #! style symbols such as #!rest. Other symbols are
 rejected by the reader. Furthermore, certain strange combinations, such as
 “(#! (foo bar))” result in an unterminated list, no matter how many close
 parens are added. The latter is the issue I am hitting.

 The fix is probably to explicitly test for valid tokens, which appear to
 be only: “optional”, “rest” and “key”. Probably “eof” is ok as well—not
 100% sure. The reader, of course, already tests for these exact tokens, so
 the writer should too.

 Alas, chicken-doc won’t work with Chicken 5 until this is fixed — I worked
 around file locking bug #1565 in 5.0.0, but the writer bug is fatal.

 Jim

--
Ticket URL: <https://bugs.call-cc.org/ticket/1572>
CHICKEN Scheme <https://www.call-cc.org/>
CHICKEN Scheme is a compiler for the Scheme programming language.