Re: [emacs-bidi] debugging visual-to-logical

[Eli Zaretskii]

On Mon, 19 Nov 2001, Alex Schroeder wrote:

> Furthermore, users usually don't see the logical representation, so
> humans won't even be in the position to judge wether a representation
> is reasonable.

I think you are wrong.  I have no problem determining that, in the 
specific case of converting "<123H/>shalom<123H>" from visual to logical, 
"<Hshalom<123</H123>" is not a reasonable result, simply because it 
doesn't look like anything a human would type for that kind of output.

> There are probably more of these extra rules around: We need
> to find them such as the eliminate "unreasonable" logical
> representations.  This might be rather hairy.

Yes.  But I'm mostly saying that I hope this will be unneeded, provided 
that all, or even most, of the conversions give ``reasonable'' results.

> The question is: "Is this logical representation reasonable for
> humans?"
> 
> Personally I don't really think so, and I think that my suggestion is
> not that good.  I remain convinced that we cannot determine what is
> "reasonable" using an algorithm.  The counter example is "my SLK230"
> where "SLK230" is a specific name, a label.  It displays as "my
> 230KLS".  You would never search for it by typing "m y 2 3 0 S L K".

The solution to this is only possible when examples are concrete (as 
opposed to randomly-abstract), and within context.  On other words, the 
text we discuss must be meaningful, not some random string.  Let me 
explain what I mean, taking the search issue we were discussing recently.

To that end, let's take the above example, but change it a bit, so that it 
has some meaning.  Suppose I want to say "my bmw230", where "bmw" is a 
car mark.  If the car mark is in Hebrew, I'd type "my BMW230" and expect 
it to be displayed as "my 230WMB".  When I later search for such a 
string, I would also type "m y   B M W 2 3 0", because THIS IS THE ONLY 
WAY I CAN GIVE A MEANING to what is displayed as "my 230WMB"--I _know_ 
than "WMB" is a car mark.

Let's now take the other example, which is "my 123CAR" in the logical 
order, and again change it to give it some meaning.  Suppose I want to 
say "my 123 cars" (I can dream, can't I? ;-), or in Hebrew "my 123 
CARS".  This will be displayed as "my 123 SRAC".  When I later search for 
this text, there's no chance I'll ever try to type "m y   C A R S   1 2 3",
because the utterance "my CARS 123" is meaningless in Hebrew.

This is how the frequent and reasonably probable situations are 
automatically resolved.  Of course, there are those rare and unreasonable 
situations where someone could type the search strings the other way 
around, and then they will not find them.  But as long as these cases are 
rare, we can simply ignore them.

I agree that a feature whereby Emacs will look for all the possible 
logical strings that produce a given visual-order string, or the same 
visual-order string as the logical-order string the user typed, would
be _very_ nice.  However, since the set of all such possibilities is 
infinite, I don't see how can we do something like that.

In the case of automated conversion from the visual to logical order, the 
above is not possible, since a program doesn't know what humans do.  But 
we should still try to tweak the mechanistic application of rules so as 
to produce a ``reasonable'' conversion, where ``reasonable'' means 
something that could have been typed in similar situations.  Since we, 
the humans, are the judge of ``reasonable'', such tuning of the algorithm 
is possible.

Please keep in mind that the visual-to-logical conversion should be 
required infrequently, and the results will probably be perused by a 
human.  So if the program does a reasonably good job, we could rely on 
the human to fix the few problematic cases.

> How about this approach: We identify the operations on the logical
> representations which have no effect on the visual representation.
> This might be easier to do than to identify potential problems and
> arguing about "resonable" or not.  Then we can write a function which,
> given a logical representation, generates various equivalent logical
> representations.

If you can think of code that can generate a finite (and not too large!) 
number of equivalent logical-order strings, please suggest how to do that.
Unless I'm missing something, there's an infinite number of equivalents, 
given the possibility of using formatting codes.