What files can we convert to UTF-8? -- was: (Re: Localization not using Unicode?)

[Davi Leal]

Laurentiu Matei wrote:
> > Maybe we should change all files to an Unicode encoding, even the
> > .php, .tpl, etc., which could have some strings embedded.
>
> It's important too to make sure the encoding of the HTML generated by
> the app is UTF-8.

* 98% of the HTML come from the .tpl Smarty templates.
* 1% comes from embedded HTML on .php files.
* 1% Others files as the images/gnus-desc.html, which
  maybe we should add the translation support to.


I have processed all the dev_1_1 files, not being directories or
images, with the below command:
 iconv --from-code=ISO-8859-1 --to-code=UTF-8  file > file.u ; mv file.u file

I was surprised only some files produced a difference from the
original. Such files are:

./AfferoGPL
./Layer-0__Site_entry_point/doc/GNUHerds__SQL_Implementation.psql
./locale/es/LC_MESSAGES/messages.po
./locale/it/LC_MESSAGES/messages.po

I run the 'file *' command to check. The below file types does not
produce a difference after executing the above 'iconv' command:
 PHP script text
 ASCII English text
 ASCII English text, with very long lines
 HTML document text
 exported SGML document text


The result is that only these files have been converted to UTF-8:
./AfferoGPL
./Layer-0__Site_entry_point/doc/GNUHerds__SQL_Implementation.psql
./locale/es/LC_MESSAGES/messages.po
./locale/it/LC_MESSAGES/messages.po

Let me know if I fix something.

P.S.: I have committed right now the changes to the dev_1_1 branch,
which we will merge to the trunk where RMS agree with it. I have added
the locale/ro/LC_MESSAGES directories.

Davi