Re: Bug in the sort utility

[Bob Proulx]

Warner Mike - miwarn wrote:
> We came across something that appears to be a bug as opposed to an
> unintended design feature.

Thank you for the bug report.  However what you are seeing is intended
behavior.  It isn't something sort has control over.  The character
collation sequence is chosen by your specified locale.

> I am expecting the output to come out in ASCII sorted order.

You don't like it and I don't like it but the-powers-that-be have
confused working with data on a computer with talking about working
with data on a computer.  They have decided that the collation
ordering (sort ordering) for data should be dictionary ordering.  In
dictionary ordering case is folded together and punctuation is
ignored.  By having LANG set to any of the "en" locales the system is
instructed to use dictionary sort ordering.  This affects almost
everything on the system that sorts.  This includes commands such as
'ls' and also your shell (e.g. 'echo *') too.

Your sort order depends upon your locale.  You didn't say what your
locale was and therefore I assume that you were not aware that it
had an effect.

The documentation says:

     Unless otherwise specified, all comparisons use the character
  collating sequence specified by the `LC_COLLATE' locale.(1)
  ...
     (1) If you use a non-POSIX locale (e.g., by setting `LC_ALL' to
  `en_US'), then `sort' may produce output that is sorted differently
  than you're accustomed to.  In that case, set the `LC_ALL'
  environment variable to `C'.  Note that setting only `LC_COLLATE'
  has two problems.  First, it is ineffective if `LC_ALL' is also set.
  Second, it has undefined behavior if `LC_CTYPE' (or `LANG', if
  `LC_CTYPE' is unset) is set to an incompatible value.  For example,
  you get undefined behavior if `LC_CTYPE' is `ja_JP.PCK' but
  `LC_COLLATE' is `en_US.UTF-8'.

Personally I have the following in my $HOME/.bashrc file.

  export LANG=en_US.UTF-8
  export LC_COLLATE=C

That sets most of my locale to a UTF-8 one but forces sorting to be
standard C/POSIX.  This probably won't work in the general case since
I have no idea how that would interact with all character sets.

> Is this a bug that has been reported and corrected already?

You may want to look at the FAQ.

  
http://www.gnu.org/software/coreutils/faq/#Sort-does-not-sort-in-normal-order_0021

Bob