|
From: | Simon Ledergerber |
Subject: | bug#20623: XML and HTML files with encoding/charset="utf-8" declaration loose BOM; Coding system is reset from utf-8-with-signature to utf-8 on save |
Date: | Fri, 22 May 2015 15:21:00 +0200 |
User-agent: | Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 |
Hello EliI have done some more research to answer your questions. You will find the details of my statement at the end of this mail.
On 22.05.2015 09:11, Eli Zaretskii wrote:
You're right. When I open a file that was really saved with BOM, Emacs detects its encoding correctly, i. e. utf-8-with-signature-dos. But when I change the content and save with C-x C-s, the encoding changes to utf-8-dos and the BOM gets lost. Even when I start Emacs with -Q. This is the actual bug.[Please don't remove the bug address from the CC list, so that this discussion is recorded in the bug data base.]Date: Thu, 21 May 2015 22:49:47 +0200 From: Simon Ledergerber <sledergerber@gmx.net> From the documentation I understand that utf-8 is without BOM and utf-8-with-signature is with BOM. Maybe I am wrong and should rather understand that utf-8 is auto-detect. But then there is something like utf-8-without-signature missing to specify explicitly that no BOM is desired. In my opinion, it is correct when Emacs prefers utf-8 over utf-8-with-signature when it opens a file without BOM that can still be recognized as UTF-8. However when a file is opened with a BOM already present, it should stick to the utf-8-with-signature coding system, because the BOM "EF BB BF" unambiguously marks the file as UTF-8. (For UTF-16 for example, there is a different BOM byte pattern. There are other coding systems which do not have a BOM at all.)What do you mean by "stick to"? When I try visiting an XML file that is encoded with BOM, Emacs decodes the file correctly, and the value of buffer-file-coding-system is utf-8-with-signature. Isn't that what you want? If that's what you want, but it doesn't happen for you, please try in "emacs -Q". It's possible that the default you set: (setq-default buffer-file-coding-system 'utf-8-dos) is the reason for what you see. (I don't understand why you need such a default, and it sounds like a bad idea to me.)
This is true and it worked for me. Please see below for further explanations.By doing C-x <RET> f and then saving it with C-x C-s, I expect to be able to change the coding system. For example, if I specify utf-8-dos, the BOM should be removed, if one was present, and CR LF should be inserted for EOL. On the other side, if I choose utf-8-with-signature-unix, a BOM should be written and LF be taken for EOL. (The conversion between DOS and Unix works, just the BOM is the problem.) I have found a link, where this topic was already discussed, but it didn't help me further: http://superuser.com/questions/41254/make-emacs-not-remove-the-bom-from-xml-files In that post Vebjorn Ljosa asked exactly the question I have. Richard Hoskins replies with the answer to change the coding system with C-x <RET> r utf-8-with-signature. Unfortunately, it didn't work for me - after doing a change in the file and saving, it got back to utf-8 automatically - that's why I have filed the bug.That's not how you force a file to be saved in a specific encoding. You should do this instead: C-x RET c utf-8-with-signature RET C-x C-s The "C-x RET c" prefix forces the next Emacs operation to use the specified encoding. In this case, Emacs will ask for confirmation, because the encoding you specified is different from what the XML comment says.
Summary:- C-x RET c utf-8-with-signature RET C-x C-s is a good workaround, because it really forces the file being written with BOM. In order to have an effect however, the file must be dirty, i. e. there must be a pending change. But before the command completes in this case, the prompt "Selected encoding utf-8-with-signature-dos disagrees with utf-8-dos specified by file contents. Really save (else edit coding cookies and try again)? (yes or no)" appears. I think this is what you mean with your sentence: "In this case, Emacs will ask for confirmation, because the encoding you specified is different from what the XML comment says."
- But consider the following: The encoding in the XML declaration or in the HTML <meta charset="utf-8"> just specifies UTF-8 (or another encoding). It doesn't say anything about the presence or absence of the BOM. Therefore an editor detecting and deciding about the file's encoding should not rely on this information only.
- When such a file, which was saved successfully with BOM, is closed and reopened again, Emacs detects its encoding correctly, say utf-8-with-signature-dos.
- However, when I change the file content and save it again just with C-x C-s (without C-x RET c ... first!), then it changes back to utf-8-dos. Yes, even if I start emacs with -Q! (That's the point.)
- I do not fully understand the criterion for and the magic behind how Emacs chooses the file encoding when I do C-x C-s. But I was able to reproduce it several times by applying the procedures given in the bug report, even when -Q is on. As we already have stated above, this could be because Emacs favors (and forces) utf-8 whenever it sees something like XML or HTML that might be UTF-8-encoded.
-> Conclusion: C-x RET c utf-8-with-signature RET C-x C-s is a good way to force the file being written as I want. But what I still do not understand: When I open a file with BOM and Emacs recognizes that, why does it change the encoding silently to drop the BOM when I regularly save with C-x C-s - and this even without giving me a notice or warning?
[Prev in Thread] | Current Thread | [Next in Thread] |