Re: [vile] utf-8 newbie

[Paul Fox]

so i tracked this down to my environment (not too surprising).  i've
set "export LC_ALL=C" in my .profile for some time, in order to get
traditional sort ordering for ls.  this LC_ALL=C tells vile i can't/don't
do wide chars.  changing to this seems to work for now:
    unset LC_ALL
    export LC_COLLATE=C

thanks for the hints.

paul

thomas wrote:
 > > resurrecting an old utf-8 thread...
 > > 
 > > i've now built vile-9.8g.  the behavior is still exactly as
 > > described below.   "locale -a" does not list en_US, but it
 > > sounded from what tom said below that that shouldn't matter
 > > after 9.8e.
 > > 
 > > so, my test uses a file containing this line:
 > >  drivers/input/mouse/psmouse-base.c: In function 
 > > ???psmouse_process_byte???:
 > 
 > hmm - the email doesn't act like utf-8 (and the charset might hint at the
 > problem).
 > 
 >      Content-Type: text/plain; charset="US-ASCII"
 >      Content-ID: <address@hidden>
 >      Content-Transfer-Encoding: quoted-printable
 > 
 >      resurrecting an old utf-8 thread...
 > 
 >      i've now built vile-9.8g.  the behavior is still exactly as
 >      described below.   "locale -a" does not list en_US, but it
 >      sounded from what tom said below that that shouldn't matter
 >      after 9.8e.
 > 
 >      so, my test uses a file containing this line:
 >       drivers/input/mouse/psmouse-base.c: In function 
 > =E2=80=98psmouse_process_=
 >      byte=E2=80=99:
 > 
 > But here (within parentheses) is your first test-character.  In uxterm I see 
 > it
 > as a left single-quote.
 > 
 > (‘)
 > 
 > In POSIX locale I see that as \xE2\x80\x98.  In en_US locale, I'd see
 > the first byte as "a" with "^" on top.  The other bytes are still in
 > hex.
 > 
 > Here's the conversion I used to check the data:
 > 
 > 0x2018: 8216 020030 0x2018 text " \030" utf8 \342\200\230
 > 0342: 226 0342 0xe2 text "\342" utf8 \303\242
 > 0200: 128 0200 0x80 text "\200" utf8 \302\200
 > 0230: 152 0230 0x98 text "\230" utf8 \302\230
 > > 
 > > i'm running uxterm, vile 9.8e, with "set fk=utf-8".  for the record,
 > > if i use uxterm and vim and "set encoding=utf-8", it displays correctly.
 > > do i need to set printing-low/high to anything special?
 > > 
 > > any other thoughts?
 > 
 > What does "locale -a" and "locale" show?  (It's been a few months since I 
 > visited
 > this area, but still...)
 >  
 > > paul
 > > 
 > > 
 > > thomas wrote:
 > >  > On Fri, 24 Jun 2011, Paul Fox wrote:
 > >  > 
 > >  > > i guess if gcc is going to start putting UTF-8 in its error messages,
 > >  > > it's time for me to figure out how to display them.  sigh.  it's been
 > >  > > a good run.  :-)
 > >  > >
 > >  > > i think my problem is simple.  simple xterm doesn't solve it, but
 > >  > > uxterm does, and the gcc error messages look fine.  running vile under
 > >  > > uxterm isn't so successful.  i unset printing-low/high hoping that the
 > >  > > defaults for everything else would do the right thing, but i still get
 > >  > > \xNN escapes:
 > >  > >    foo.c: In function \xE2\x80\x98ec_write\xE2\x80\x99:
 > >  > > that's true with file-encoding set to locale (the default), auto,
 > >  > > or 8-bit.
 > >  > >
 > >  > > if i change file-encoding to "utf-8" i get:
 > >  > >    foo.c: In function \u2018ec_write\u2019:
 > >  > > which maybe is better, but i think i'm on the wrong track.
 > >  > >
 > >  > > i have LC_ALL=C and LANG=en_US.UTF-8 set.  i'm running vile-9.7zd
 > >  > > (because that's what came with ubuntu maverick.  i can build a newer
 > >  > > version if that will help).
 > >  > 
 > >  > perhaps this - easy to check:
 > >  > 
 > >  > until 9.8d/e, vile would rely upon having (installed in the system) 
 > > locale 
 > >  > data for en_US and en_US.UTF-8, which was workable for several years 
 > > until 
 > >  > Ubuntu (and others, though iirc, Ubuntu was the first) reduced their 
 > >  > locale support.
 > >  > 
 > >  > In 9.8d, I added a builtin table (about 70kb) to provide that 
 > > information
 > >  > (but there was a remaining bug that I fixed in 9.8e).
 > >  > 
 > >  > If "locale -a" doesn't list en_US, that's the first place to consider.
 > >  > 
 > >  > On other fronts, someone's packaged vile for Fedora, but (the last I 
 > >  > checked a couple of weeks ago), it's got a problem with the library
 > >  > path, making most syntax filters fail to load.  It's a nuisance when I 
 > >  > update with yum, since my working package gets overwritten.  So I have 
 > > a 
 > >  > to-do to write a comparable (but working) rpm spec...
 > >  > 
 > >  > At the moment I'm working on dialog, expecting to go to xterm next - 
 > > based 
 > >  > on how big my backlog is - sorry for being slow to get back to vile 
 > >  > (dialog has a lot of work due to the recent adoption in FreeBSD).
 > >  > 
 > >  > -- 
 > >  > Thomas E. Dickey
 > >  > http://invisible-island.net
 > >  > ftp://invisible-island.net
 > > 
 > > =---------------------
 > >  paul fox, address@hidden (arlington, ma, where it's 56.8 degrees)
 > > 
 > 
 > > _______________________________________________
 > > vile mailing list
 > > address@hidden
 > > https://lists.nongnu.org/mailman/listinfo/vile
 > 
 > 
 > -- 
 > Thomas E. Dickey <address@hidden>
 > http://invisible-island.net
 > ftp://invisible-island.net
 > -----BEGIN PGP SIGNATURE-----
 > Version: GnuPG v1.4.9 (GNU/Linux)
 > 
 > iEYEARECAAYFAk+Il88ACgkQcCNT4PfkjtumHACeKAJVjpdtsKWE5ozLDyYFDeOs
 > FF0AoID7/juFYAcBFi73fbexWUYvHZoi
 > =0xe6
 > -----END PGP SIGNATURE-----

=---------------------
 paul fox, address@hidden (arlington, ma, where it's 57.4 degrees)