Re: mhfixmsg character set conversion

[Steven Winikoff]

>The file has UTF-8 and later ISO 8859-1.

Another point that should have been obvious to me, and is in hindsight, is
that I can't expect vim to detect the character set properly for something
like this. :-/


>There's no BOM so ucs-bom fails.  The ISO 8859-1 bytes don't happen to
>be valid UTF-8.  ‘default’ means use your environment, which is probably
>UTF-8 again; fails.  Which means we arrive at ‘latin1’, AKA ISO 8859-1,
>which is happy.

Happy, and just as half-correct as utf-8 would have been.

Meanwhile, I did a web search based on what you wrote here, and discovered

   https://vim.fandom.com/wiki/Working_with_Unicode

...which confirms everything you wrote, but also

   https://stackoverflow.com/questions/25115752/vim-encodings-latin1-and-utf-8

...which suggests using this command in vim to force it to reload the file
in utf-8 encoding:

   :e ++enc=utf-8 path_to_file

Of course this can also be done directly from the command line as

   vim -c "e ++enc=utf-8" path_to_file

More interestingly, when vim reopens the file (or just opens it, in the
latter case) in utf-8, it emits this status line message:

   "/tmp/nmh_testing/bad" [ILLEGAL BYTE in line 289] 336 lines, 49366 bytes

...and of course the line in question contains accented characters encoded
in ISO 8859-1, so everything is consistent.


>> ...but in bash, although the line gets pasted, the newline at the end
>> of it somehow doesn't.
>
>Another difference is the pasted text is normally highlighted in some
>way, e.g. inverse video, until it's committed with Enter.

In my experience with tcsh, the inverse video highlighting stays in place
even after the paste is committed, and remains so until something else is
highlighted.

This appears to be the case for me in bash (invoked as sh just now), at
least with enable-bracketed-paste turned off.

     - Steven
-- 
___________________________________________________________________________
Steven Winikoff      | "Science is built upon facts, as a house is
Montreal, QC, Canada |  built of stones; but an accumulation of
smw@smwonline.ca     |  facts is no more a science than a heap of
http://smwonline.ca  |  stones is a house."
                     |                           - Henri Poincaré