Re: [cjk] Fwd: HELP with emacs 25.1 and cjk-enc.el

[Bo-Yin Yang]

Dear Hin-Tak,

I think I mostly set LC_CTYPE and LC_CTYPE, when outside of emacs, I set Options -> Multilinguial Environment -> Set Language -> Chinese -> Big5

before I compile in emacs ... 

The following is my "cjklatex" batch file with macOS with an ancient port (emacs 24.4.1)

#!/bin/sh

export LANG=zh_TW.Big5

export LC_ALL=zh_TW.Big5

export LC_CTYPE=zh_TW.Big5

f=`echo $1 | sed -e 's|\(.*\)\.[^/]*$|\1|'`

echo "converting $1 to $f.cjk"

/Applications/MacPorts/Emacs.app/Contents/MacOS/Emacs -batch -l cjk-enc -f batch-force-cjk-write-file $f.tex 2> /dev/null

latex $f.cjk

and in Fedora 20 (emacs 24.3.1)

#!/bin/sh

export LC_CTYPE=zh_TW.Big5

export LC_ALL=zh_TW.Big5

f=`echo $1 | sed -e 's|\(.*\)\.[^/]*$|\1|'`

echo "converting $1 to $f.cjk"

emacs -batch -l cjk-enc -f batch-force-cjk-write-file $f.tex 2> /dev/null

latex $f.cjk

I tried my examples with bg5latex and they do in fact return pdf files with bunch of numbers in them

On Sun, Dec 18, 2016 at 1:34 AM, Hin-Tak Leung <address@hidden> wrote:

> I am aware of
 bg5latex/bg5pdflatex, and just went back to check if it
 works with the test input I gave.  It does not work insofar
 as all the chinese input (in big5) become numbers. 

Becoming mostly numbers is the correct behaviour. Try running the output through (pdf)latex and see.

> I checked the Big5.tex example
 file and it seems like you have to manually add all the
 \begin{CJK}{Bg5}{bsmi} .... \end{CJK} pairs, what
 bg5latex does (as I remembered from long time ago) is
 preventing Big5 text being mangled inside the
 \begin{CJK} ... \end{CJK}.  I guess the point of
 cjk-enc.el is precluding the need for such manual
 addition.

I don't think it is necessary to add all the \begin{CJK}\end{CJK}. But emacs has become a lot more locale-aware (or locale-sensitive...)
from 23.x onwards, and getting more so for every release since. The message() breakage with emacs 25 is certainly one of these.

What would really help recent emacs is declaring encoding at the end with a command block like the example.

%%% Local Variables:
%%% coding: ...
%%% End:

So I am concentrating on Big5.tex from the cjk distribution for testing at the moment.


> I also tried your patch
 adding  (let ((coding-system-for-write
 locale-coding-system))
 which in fact gets rid of
 the running percentage reports, but still  somehow gets
 large numbers of

% \ifx\CJKpreproc\undefined\def\CJKpreproc{cjk-enc}\RequirePackage{MULEenc}\fi

I am aware of that. The message() fix fixes Thai and mule encoding only; Strangely enough, I cannot currently process big5 input with cjk-enc.el from emacs 23 onwards. When I updated cjk-enc.el for emacs 23 a few years ago, I may or may not have tested it against big 5 input, as I always used bg5latex/latexb5 and don't use cjk-enc.el for that purpose. But I remember at least getting the big5 /traditional Chinese portion of mule input to work correctly with emacs 23 was rather difficult, as emacs 23+ really want to (mis-)process things as utf-8.

Can I get your full LANG/LC_* environment? i.e. the whole of

export | grep LANG
export | grep LC_

That you can with emacs 24 and I cannot, may be a useful piece of information.

At the moment I am suspecting that pre-write-conversion/cjk-encode in cjk-enc.el needs to finish differently (an additional nil, etc) as  inserting "\ifx\CJKpreproc\undefined\def\CJKpreproc{cjk-enc}\RequirePackage{MULEenc}\fi"
at the beginning of the output is the LAST THING it should do. Somehow it is not stopping at that point, and keep recursing a few more times.

Earlier emacs seems to be more tolerant of pre-write-conversion/cjk-encode not starting/finishing the right way (saving/restoring buffer before and after); There is a emacs 25 incompatible change about

'save-excursion' does not save&restore the mark any more.
Use 'save-mark-and-excursion' if you want the old behavior.

There is a save-excursion just before a write-region, which calls pre-write-conversion.

test.pdf
Description: Adobe PDF document