Re: lynx-dev UTF-8 display questions (was: Superscripts)

[Klaus Weide]

On 8 Jun 2000, Sergei Pokrovsky wrote:

> >>>>> Klaus Weide writes:
> ...
>   Klaus> You are running into one of the fundamental problems of
>   Klaus> displaying UTF-8 with a display library that is not
>   Klaus> UTF-8-aware.  The display library (ncurses in your case,
>   Klaus> iirc) makes the assumption that one byte == one character
>   Klaus> (position).  So it would wrap a line (or possibly truncate
>   Klaus> it) after 80 characters in a 80x25 window.
> 
> I really had install fresh ncurses and xterm; but lynx didn't want
> to compile with ncurses (at least, not the native cc, which had many
> objections against the lynx code, like ; after cpp-expanded structure

Maybe Tom is interested in pursuing this...

> declarations etc), so I did configure --with-screen=slang even before
> my complaint.
> 
>   Klaus> That means that, for lines full of UTF-8 characters in a
>   Klaus> range where each character is encoded as two bytes (which
>   Klaus> includes Cyrillic), only about half of the available
>   Klaus> horizontal display width is usable.
> 
> Well, that may be the reason why some innocent tables are not
> formated, if they contain a few accented letters.

Well, only "a few" accented letters shouldn't make much of a difference;
each should only add 1 to the number of "unusable display positions",
if they are in the Unicode range U+00A0 - U+07FF.

If you can increase the horizontal window size, you should be able to
verify that you are really seeing the effect I mean:  Make the window
larger and larger (maybe reload if necessary), eventually "it shoud fit".

>   Klaus> There is one existing workaround, but only if you compile
>   Klaus> lynx with slang instead of (n)curses: compile with
>   Klaus> SLANG_MBCS_HACK defined.  For example, (this is the way I
>   Klaus> pass additional flags to the compile process)
> 
>   Klaus>    ./configure --with-screen=slang [...]
>   Klaus>    make SITE_DEFS="-DSLANG_MBCS_HACK"
> 
>   Klaus> It works well for me in most $TERMinal types (but not all -
>   Klaus> although those aren't UTF-8 capable anyway).
> 
> I've tried that -- to no avail.  Nothing is changed.

That is strange.  Does -DSLANG_MBCS_HACK get passed to the cc invocation?
Which version of the slang library are you linking to?

> I must admit though that I've simply removed the object files and
> rerun ./configure --with-screen=slang; for some reason
> "make distclean" removes too much and no ./configure is possible after
> it;

I haven't tried this (at least not recently).  It sounds like a bug,
but maybe "make clean; rm config.cache" instead of "make distclean"
acts better?

> and I wouldn't like to restart everything form the tarball, given
> the amount of massaging necessary to make the sources pass through the
> Solaris cc.

For this (-DSLANG_MBCS_HACK), you should only have to regenerate
src/LYCurses.o and src/LYUtils.o, if you already had a configuration
with slang that works.  No re-running of ./configure should be necessary.

I haven't tried this with the latest and greatest sources, but I am not
aware of changes in this area that would prevent it from working.
> 
>   Klaus> With the advent of an UTF-8-aware ncurses, this problem may
>   Klaus> be solved soon (for those who have it) (after code changes in
>   Klaus> lynx).
> 
> I'd like to live to see it :)

But you'd have to be able to compile with ncurses...

  Klaus


; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden