Re: lynx-dev Emacs + UTF-8 + Lynx

[Sergei Pokrovsky]

>>>>> "Klaus" == Klaus Weide <address@hidden> writes:

  Klaus> On 25 May 2000, Sergei Pokrovsky wrote:

...

  Klaus> What is browse-url-lynx-emacs?

Here's the beginning:

,----
| (defun browse-url-lynx-emacs (url &optional new-buffer)
|   "Ask the Lynx WWW browser to load URL.
| Default to the URL around or before point.  With a prefix argument, run
| a new Lynx process in a new buffer.
| 
| When called interactively, if variable `browse-url-new-window-p' is
| non-nil, load the document in a new lynx in a new term window,
| otherwise use any existing one.  A non-nil interactive prefix argument
| reverses the effect of `browse-url-new-window-p'.
| 
| When called non-interactively, optional second argument NEW-WINDOW is
| used instead of `browse-url-new-window-p'."
|   (interactive (browse-url-interactive-arg "Lynx URL: "))
|   (let* ((system-uses-terminfo t)       ; Lynx uses terminfo
|          ;; (term-term-name "vt100") ; ??
|          (buf (get-buffer "*lynx*"))
|          (proc (and buf (get-buffer-process buf)))
|          (n browse-url-lynx-input-attempts))
|     (if (and new-buffer buf)
|         ;; Rename away the OLD buffer. This isn't very polite, but
|         ;; term insists on working in a buffer named *lynx* and would
|         ;; choke on *lynx*<1>
|         (progn (set-buffer buf)
|                (rename-uniquely)))
|     (if (or new-buffer
|             (not buf)
|             (not proc)
|             (not (memq (process-status proc) '(run stop))))
|         ;; start a new lynx
|         (progn
|           (setq buf
|                 (apply #'make-term
|                        `("lynx" "lynx" nil ,@browse-url-lynx-emacs-args 
,url)))
|           (switch-to-buffer buf)
|           (term-char-mode)
|           (set-process-sentinel
| (...)
|     
`----

  Klaus> With which command line options does it call lynx?

,----
| (defcustom browse-url-lynx-emacs-args (and (not window-system) 
|                                            '("-show_cursor"))
|   "A list of strings defining options for Lynx in an Emacs buffer.
| 
| The default is none in a window system, otherwise `-show_cursor' to
| indicate the position of the current link in the absence of
| highlighting, assuming the normal default for showing the cursor."
|   :type '(repeat (string :tag "Argument"))
|   :version "20.3"
|   :group 'browse-url)
`----

  Klaus> If you call lynx non-interactively, i.e. with -dump, and then
  Klaus> cat the results in a similar environment - are the characters
  Klaus> correct?

Yes.  I've done this:

,----
| (let ((coding-system-for-write 'utf-8)
|       (coding-system-for-read 'utf-8))
|    (term "/usr/local/bin/bash"))
`----

and in this bash session (which receives and outputs in utf-8) I've
said

,----
| 16:35:05 pok#5$ lynx -dump http://www.esperanto.mv.ru/KompLeks/UTF8/A.html
`----

I'got the correct output, e.g.:

,----
|    absolut·a erar·o —
|    
|    [22]Absoluta valoro de [23]ekarto; se nombron (aŭ formulon, funkcion)
|    x oni [24]aproksimas per a, tiam la absoluta eraro estas |a−x|. Kp
|    [25]relativa eraro.
|    
|    Angle: absolute error
|    Ruse: абсолютная погрешность
|    
`----

That is, both em-dash and the Russian SMALL R behave as desired.

...

  Klaus> Note the common pattern: in both cases, the UTF-8 encoding
  Klaus> contains a 0x80 byte.  Possibly this byte value triggers some
  Klaus> error, either in emacs' terminal emulator or in the curses or
  Klaus> slang library your lynx binary is using.  I don't think the
  Klaus> error is in the lynx code, it's hard to see why it would get
  Klaus> just that byte value wrong (and if it's lynx itself, I should
  Klaus> see the error, too.)

So the error is specific to the interactive mode ...

  >> 2) When the cursor passes through an anchor, some two-byte
  >> characters are doubled; e.g. the line

...

  Klaus> The first thing to try is a newer lynx version.  Get
  Klaus> 2.8.3rel.  There are substantial changes in how UTF-8 output
  Klaus> is handled, they might solve your problem 2).

I've done that.  The error has gone.  (Also the table emulation is
improved.)

...

  Klaus> Please let us know whether 2.8.3 improves things for you.

I've found 2.8.4 and installed it.  The problem (2) has gone.

  Klaus> If your problems remain, a possible workaround is a modified
  Klaus> terminfo / termcap description.  Basically with less escape
  Klaus> sequences for cursor positioning, so curses / slang has less
  Klaus> opportunity to optimize cursor movement (and thereby mess up
  Klaus> UTF-8 output).  If you are interested in pursuing that, send
  Klaus> your terminal description (for whatever $TERM lynx sees when
  Klaus> invoked your way), and I can suggest some changes.

Is the following what you mean?  (called within the same session):

,----
| 16:58:54 pok#7$ infocmp -I $TERM
| #     Reconstructed via infocmp from file: 
/usr/local/share/emacs/20.6/etc//e/eterm
| eterm,
|         am, mir, xenl,
|         colors#8, cols#80, lines#24, pairs#64,
|         bel=^G, bold=\E[1m, clear=\E[H\E[J, cr=\r,
|         csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=\b,
|         cud=\E[%p1%dB, cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C,
|         cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
|         dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM, dl1=\E[M,
|         ed=\E[J, el=\E[K, el1=\E[1K, home=\E[H, ht=\t,
|         ich=\E[%p1%d@, il=\E[%p1%dL, il1=\E[L, ind=\n,
|         invis=\E[8m, kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC,
|         kcuu1=\EOA, kend=\E[4~, khome=\E[1~, knp=\E[6~,
|         kpp=\E[5~, rev=\E[7m, rmcup=\E[2J\E[?47l\E8,
|         rmir=\E[4l, rmso=\E[m, rmul=\E[m,
|         setab=\E[%p1%{40}%+%dm, setaf=\E[%p1%{30}%+%dm,
|         sgr0=\E[m, smcup=\E7\E[?47h, smir=\E[4h, smso=\E[7m,
|         smul=\E[4m,
`----


-- 
Sergei

; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden