Looking at a convenient issue of the POSIX standard, I find
http://pubs.opengroup.org/onlinepubs/007908799/xbd/envvar.htm, and in
particular where it mentions a "precedence order". It seems that the
interaction of environment variables is not unspecified at all.
I'm guessing that something else was actually meant: the writer perhaps
found it hard to describe in a general way the interaction of a collation
locale with the contents of a file to be sorted if it happens that the
contents were created in another locale and are to be interpreted in the
way they were created. So, applying LC_COLLATE=C to Chinese big-5 could
well produce a peculiar order of things.
This has already been the case for me. My data is ASCII, comprising
numeric data and type data, both normal and encoded. All codes are
printable ASCII and are specifically designed to be sorted based on the
contents of bytes. This did not work well with LANG=en_US.UTF-8, because
for this data 'a' and 'A' are numbers that differ by 26, but the LANG
setting was treating them as nearly equivalent. It seems that LC_COLLATE=C
is the correct cure.
If I'm right, it seems that it would be better to rewrite that footnote in
the sort info page something like this:
(1) The collation order used by 'sort' is controlled by environment
variables in accordance with the POSIX specification. In particular, the
first of the LC_ALL, LC_COLLATE, and LANG variables that is defined to a
non-null value controls the collation order; in their absence your system
has a default. If the collation order is incompatible with your data, you
are unlikely to get the desired results. Often, but not always, setting
and exporting LC_COLLATE=C in your environment is the right choice, but if
your data contains natural language text or proper names the right choice
will agree with the encoding used for the data. Setting LC_ALL can be
unwise because it can affect many other things as well, such as the
language used for error and help messages. See
http://pubs.opengroup.org/onlinepubs/007908799/xbd/envvar.html or any
later version for more information.
You may also want to change the warning in the output of "sort --help" to
*** WARNING ***
The locale specified by the environment affects sort order. For correct
operation it must be compatible with your data.
Set LC_COLLATE=C to get the traditional sort order that uses native byte
values.
On Fri, Nov 2, 2012 at 11:07 AM, Bob Proulx <address@hidden> wrote:
Kevin O'Gorman wrote:
(reformatted and numbered)
A, In that case, set the `LC_ALL' environment variable to `C'.
B. Note that setting only `LC_COLLATE' has two problems.
B1. First, it is ineffective if `LC_ALL' is also set.
B2. Second, it has undefined behavior if `LC_CTYPE' (or `LANG', if
`LC_CTYPE' is unset) is set to an incompatible value.
B2x. For example, you get undefined behavior if `LC_CTYPE' is
`ja_JP.PCK'
but `LC_COLLATE' is `en_US.UTF-8'.
The example in B2x is illogical since A and B together mean we're
setting
LC_COLLATE to C, not some random value like en_US.UTF-8.
I want to know if LC_COLLATE=C can be messed up by an LC_CTYPE setting,
or
anything besides LC_ALL. I'm writing software that will use sort
extensively in unknown environments, and I'd like to keep all
adjustments
as localized as possible. So far, setting the collating sequence to
POSIX
is all that I need; no other locale adjustments.
I also agree that the above is needlessly disjoint. It doesn't flow.
Would you be able to suggest an improvement to the wording that would
make it better than the current prose? Of course a submission as a
patch would be great. Using git patch submissions is the preferred
format. But just saying what you think it should say would also be
appreciated.