Re: lynx-dev Emacs + UTF-8 + Lynx

[Thomas Dickey]

On Sat, May 27, 2000 at 03:17:03PM +0700, Sergei Pokrovsky wrote:
> >>>>> Thomas Dickey writes:
> 
> ...
> 
>   Thomas> I put together a crude version of Unicode support for
>   Thomas> ncurses last week, which I (or any interested person) may
>   Thomas> use to test an integrated variation of UTF-8 support.
>  
> Well, I've installed both xterm and ncurses from your Web page.  There
> is a great improvement.
> 
> The Lynx presentation within Emacs is almost ideal now.  The only
> slight defect is the ragged background: if I specify for Emacs a
> non-white background, every line that contains multibyte characters
> becomes that much shorter; and as I normally prefer the black
> background, I see a black comb at the right side of the white Lynx
> screen.  That's ugly but tolerable (of course if I select a white
> background in the Emacs settings the two backgrounds become
> indistinguishable; but I'd rather prefer to be able to have dark
> background color pattern in Lynx as I can have it in Emacs, and an
> easy way to commute among them).

This sounds like a problem managing the clearing of backgrounds.  I was talking
recently with someone who is working on Emacs, and it sounds like he is working
on that problem.  As I understand it, the term support in Emacs uses a termcap
interface to render the display something like the 'screen' program does.  So
while Lynx may be doing the "right thing", the Emacs term interface may not
render it properly.  (Mostly 'screen' does the right thing for me, though it
doesn't work well for the line-drawing characters when I test 'dialog' or
'cdk').
 
> The situation is worse with the stand-alone xterm.
> 
> After I've installed xterm, I've got a nice colored display, which
> rendered most of UTF-8 almost correctly when called with -u8 (except
> for the 0x80 bit bytes mentioned earlier).

I didn't notice the mention of the 0x80
> 
> Than I've installed the ncurses, having supplied the ./configure
> script with --enable-widec (maybe that was an error?).

no - it's ok.  But the resulting ncurses library is not compatible with
applications that may have been compiled previously.  I'm planning to modify
the --enable-widec option to also rename the resulting libraries (e.g.,
libncurses.so would be libncursesw.so).  When I originally started working
on the --enable-widec logic, I had anticipated making that ncurses 6.0,
but I'm finding now that wide characters should be more than just 16-bits.
So this is going to be just an in-between implementation (uses less memory
than the final implementation, but has limitations).

I should have implemented the UTF-8 driver before, but had no immediate need
for it.  (I'll also review some to be sure that the driver is working
properly).

> Now the Unicode characters are rendered better in xterm -u8, the
> Russian small R no longer misbehaves, I do see em-dash.
> 
> *But*
> 
> 1) The fi ligature is rendered as a box in the anchors (see the anchor

I'll look at this.  I'm not sure whether a box is a character that does
not appear in the font, or if it is a multi-plane character (requires merging
of 2 or more Unicode values).  There is an experimental patch to handle that,
but I've not integrated it yet (it needs more work).

>    "Bibliografiaj referencoj" in the next to last line of
>    <http://www.esperanto.mv.ru/KompLeks/UTF8/HEJMO.htm>: it is
>    rendered as "Bibliogra[]aj referencoj" in xterm, but as the
>    expected "Bibliogra?aj referencoj" in Emacs Lynx).  The same
>    problem in the line
> 
>                       E, F, G, ?, H, ?, I, J, ?
> 
>    where the anchor ? is rendered as | (the Latin-1 equivalent of the
>    same letter from Latin-3?).  In the plain text that letter is rendered
>    as expected, e.g.
> <http://www.esperanto.mv.ru/KompLeks/UTF8/UNIKODO.html#La-Unikoda-pagxo-Euxrope-Latina>
> 
> 2) Lynx has lost the colors.

This is a symptom of the incompatibility that I mentioned.  The bits in a
chtype that represent color are shifted by 8 bits.  The bold and underlining,
etc., also are shifted.
 
> All that isn't very important for me, as it is much more convenient to
> launch Lynx within Emacs and enjoy its advantages (like copy-paste to
> and from the Lynx buffer, some more familiar keys etc).  But maybe it
> would help in debugging process.
> 
> -- 
> Sergei
> 
> ; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden

-- 
Thomas E. Dickey <address@hidden>
http://dickey.his.com
ftp://dickey.his.com

; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden