Keepun wrote:
Files with
encoding greater than 8 bits without BOM at the beginning can
be immediately identified as binary.
No, the BOM is not required or recommended in UTF-8, so it would
be a mistake to identify GNU/Linux text files as binary merely
because they lack a BOM. Typically these files do not have a
BOM, and when they do one of the first things many users do is
remove the BOM because it can cause trouble in practice.
Diffutils does not support UTF-16, where a BOM would make more
sense, and there are no plans to add support for UTF-16 (or for
UTF-32, for that matter).