Re: index sorting in texi2any in C issue with spaces

[Gavin Smith]

On Sun, Feb 04, 2024 at 10:27:00PM +0100, Patrice Dumas wrote:
> > > However, if there is a
> > > possibility to get variable elements set to "non-ignorable" in C,
> > > possibly by using an hardcoded locale of en_US, it will not possible to
> > > get automatically both the correct and more rapid option.  The user
> > > would still have to set COLLATION_LOCALE to get it.
> > 
> > If this is possible, then we silently switch to using the C sorting 
> > if we can detect that we can treat variable elements in such a way.
> > This would be the "default", non-tailored collation.
> 
> Ok.  In that case, it is even clearer to me that COLLATION_LOCALE is not
> really useful, I think that it will be quite challenging to understand
> what it really does and will probably be unused.  I think that we should
> not propose COLLATION_LOCALE at all to users.  It seems that we should
> consider it as a short-term way to be able to use collation in C, that
> may not be worth having in the long term, and keep it as a feature
> documented as being only for XS and only for development/testing to
> allow developers to test what output it leads to and that it could be
> removed at any time.  Also calling it something else, like
> XS_STRXFRM_COLLATION_LOCALE.
> 

That's fine by me, either accessing it with an obscure testing variable
or not offering it at all.

> > There wouldn't any harm in implementing it as an option.  We'd have to
> > decide if it went via strxfrm_l, Unicode::Collate::Locale, or configurable
> > for either.
> 
> I think that we should decide it now in order to have a fully specified
> interface, even if it is not fully implemented.  To me it would seems
> much more logical to have it be similar to COLLATION_LANGUAGE as it is
> the correct one.

Makes sense, that way we can avoid being stuck with bad decisions.  Could
it be DOCUMENTLANGUAGE_COLLATION, so

texi2any -c DOCUMENTLANGUAGE_COLLATION=1

Is there not a customization variable called "documentlanguage" as well?
One possibility is to get rid of COLLATION_LANGUAGE and do it through
"documentlanguage", like

texi2any -c DOCUMENTLANGUAGE_COLLATION=1 -c documentlanguage=se

instead of

texi2any -c COLLATION_LANGUAGE=se

This would not be as flexible, though, as documentlanguage affects
other things (e.g. translations, what language the output is marked
as in some output formats).

Can we also think if there is an alternative to the word "language"
in the name of COLLATION_LANGUAGE?  The manual for Unicode::Collate::Locale
gives a possibility of "de__phonebook" and "German phonebook" does not
count as a language.  COLLATE_RULES?  COLLATE_TAILORING?  Maybe
COLLATE_LOCALE?  (The last matches the usage of Unicode::Collate::Locale.
This is disregarding the meaning of COLLATE_LOCATE that I previously
suggested).