Re: [emacs-bidi] status? news?

[Roozbeh Pournader]

On Fri, 19 Jan 2001, Eli Zaretskii wrote:

> > When the user types the exclamation mark while being in English keyboard
> > mode, the editor finds that the exclamation should be treated L2R.
> 
> That's possible, but it has a couple of disadvantages:

That has been the reason I'm suggesting the visual buffer. (I agree that
logical buffers are also needed, but they are much harder to implement
smoothly, I believe).

>   - If the user inserts an exclam into an already existing text, and
>     if the current input method is not for a R2L language, the exclam
>     will go to a wrong place.  This might confuse users, because they
>     don't expect a key such as `!' to behave differently in different
>     situations.

She should learn to be aware of the keyboard mode. Due to my experience,
she will adapt to this soon. It's much simpler than understanding LRM and
RLM. I also think that the exclam should never go to a wrong place. When
in logical mode, the behaviour should be different when the user is typing
a character sequence and ends that with an exclamation mark, and when
she's inserting a exclamation mark in the place. Let's look at the
example again:

          typing order                          desired visual display
          ------------                          ----------------------
HE SAID "it is a car!" AND RAN      =>      NAR DNA "it is a car!" DIAS EH

She has something like

        NAR DNA "it is a car" DIAS EH
                2           1

on her display, places the cursor between 'r' and '"' no 1, and types the
exclamation mark. The editor really has

        HE SAID "it is a car" AND RAN
                1           2

in its logical buffer, and sees that the place between 'r' and '"' maps to
two different betweens (!) in the buffer, between '"' no 1 and 'i', and
between 'r' and '"' no 2. But which should it select? Is the exclam really
an R2L exclam that should appear after the first quotation, or an L2R one
that should appear after 'r'?

It really needs to know about the keyboard mode, because the keyboard mode
really makes a difference.

>   - Text which comes from a source that is not the keyboard cannot be
>     treated that way.  And since not all applications support RLM and
>     other Unicode special characters, this is a real problem.

If the source is a file encoded in Unicode, 8859-6 or logical 8859-8,
Unicode algorithm should be applied, to get the text displayed the same
way everywhere. If it's coming from a place that another protocol
exists (like visual 8859-8, ISIRI 3342, or Mac Farsi), text should
converted and marks inserted to come out right. You will need to be as
much intelligent as possible in this case. The exclamation mark will be
put in the right place in this phase.

About cutting and pasting, it really depends on the OS, doesn't it? If
neither the source application nor the OS know about bidi, we can't
guarantee anything. We can do something, but the situation will be the
same as importing a file.

BTW, about the OS-level support, there has been some discussions in
linux-utf8 recently.

> Therefore, I'd prefer this case to be handled correctly
> automatically.  I cannot understand why doesn't UTR9 handle the exclam
> like this by default.  Can you tell?

Yes. Exclamation mark is simply a netrual character. Any sequence of
neutral characters that have characters of different directions on left
and right, take the line direction, which is R2L in the example. (I'm not
talking exactly in Unicode bidi terms).

That's the same rule that renders the quotation marks R2L. It really
cannot find that the exclam should be L2R, that it should stick to the
'r'. Doing that level of intelligence, would require a much more complex
algorithm.

> > If the buffer is visual, it checks the embedding levels of the two
> > adjacent characters.
> 
> Why would you need embedding in a visual buffer?

It's the problem of inserting the exclamation mark again. I want to save
the exclam correctly in the logically order file. I need to know its
embedding level, so when I see it between 'r' and '"', I will know that it
should be stored after 'r' (with an additional LRM), and not after '"'.

--roozbeh