bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings

[Paul Eggert]

On 06/10/2017 12:24 AM, Eli Zaretskii wrote:
> So your proposal would mean a change to the Lisp reader to support
> such escapes, right?  If so, isn't such a change
> backward-incompatible?

Yes, but only in the sense that undocumented escapes evaluate to 
themselves, e.g., "\F" is currently the same as "F" in Emacs Lisp 
because there is no escape sequence \F currently defined for character 
constants. But there's nothing new here, e.g., when we added "\N{...}" 
last year we changed the interpretation of the formerly-undocumented \N 
escape.

> > Also, display-raw-bytes-as-hex would cause raw bytes to be displayed with this
> > new X escape, rather than with with the x escape.
> It could only do that for codepoints below 256 decimal, so that
> limitation should be taken into account when deciding on the proposal.

Ouch, I hadn't thought of that.

Wait -- doesn't that mean that "display-raw-bytes-as-hex" is a 
misleading name, because it affects the display not only of raw bytes, 
but of other undisplayable characters? Shouldn't we change its name to 
something more generic and more accurate, like "display-characters-as-hex"?

Anyway, to address the point you raised: how about a different idea? We 
extend the existing \x syntax in strings so that \x{dddd} has the same 
meaning as "\xdddd", except that the "}" terminates the escape. This 
syntax is used by Perl and so is in the same family as \N{...}. We also 
change display-raw-bytes-as-hex to use this new syntax when a character 
is immediately followed by a hexadecimal digit. That way, most 
characters are displayed as before, but my problematic example is 
displayed as "x\x{90}5y", which is a good visual cue of the unusual 
situation.