Re: index sorting in texi2any in C issue with spaces

[Gavin Smith]

On Tue, Feb 13, 2024 at 05:37:03PM +0100, Patrice Dumas wrote:
> On Mon, Feb 05, 2024 at 06:14:16PM +0000, Gavin Smith wrote:
> > On Sun, Feb 04, 2024 at 10:27:00PM +0100, Patrice Dumas wrote:
> > > removed at any time.  Also calling it something else, like
> > > XS_STRXFRM_COLLATION_LOCALE.
> >
> > That's fine by me, either accessing it with an obscure testing variable
> > or not offering it at all.
> 
> This is implemented (only in XS), but not documented.  Should it be?

I am happy not documenting it at all, as that makes it less likely
that people rely on it, and then it is easier for us to remove it in
the future.

> As a side note, there is a test in optional tests
> other/index_collation_test_collation_locale_sv that tests
> XS_STRXFRM_COLLATION_LOCALE with sv_SE.utf8, which seems to exist on
> my debian, as /usr/lib/locale/sv_SE.utf8/LC_COLLATE, but the linguistic
> rules implemented in Unicode::Collate::Locale for sv do not seem to be
> used.  I also think that it does not matter as this interface is
> unlikely to be used.

Maybe a bug in the Swedish locale (from glibc?) if it is not using
correct collation rules for that language?

> As another side note, it should not be too difficult to add other
> collation possibilities in the XS/C code for other platforms in
> tp/Texinfo/XS/main/manipulate_indices.c by modifying get_sort_key and
> setup_collator, possibly adding another collation type/another
> customization variable, if people wanted to contribute code to support
> their platform specific interfaces that were described in the thread by
> Eli.  I do not think that it is important, though, as the current default
> of using Perl modules leads to correct output, even if it is probably
> slower than what could be obtained with a full C implementation.
> 
> > > I think that we should decide it now in order to have a fully specified
> > > interface, even if it is not fully implemented.  To me it would seems
> > > much more logical to have it be similar to COLLATION_LANGUAGE as it is
> > > the correct one.
> > 
> > Makes sense, that way we can avoid being stuck with bad decisions.  Could
> > it be DOCUMENTLANGUAGE_COLLATION, so
> > 
> > texi2any -c DOCUMENTLANGUAGE_COLLATION=1
> 
> This is implemented

Thanks for doing this.

> , added to NEWS (could be revised for language), not
> in the manual.  I think that it would be better if you added it to the
> manual as I will not write it well.

Done.

> Regarding the alternative names, no problem with changing the names now,
> just tell me.

I think the names are fine as they are now but we could change them if
there was a better proposal before we make a release.