[emacs-bidi] reordering based on implicit levels

[Alex Schroeder]

I finished the code for the reordering function.  If you are
interested in the code, send me mail.  It's about 30k right now.  You
need at least one table.  The testing table is around 5k, the real
(and still incomplete) one is 545k.  :)

I'd be interested in a list of test cases -- obviously only test cases
which are based on implicit levels.  I'm not sure what the next steps
are.  Things which I might want to add is refilling of such thrings,
working on paragraphs, adding explicit levels (should another
application put the chars in visual order into the clipboard and leave
the explicit direction codes in there -- don't know wether such an app
actually exists), trying to do the right things if being fed chunks of
bidi text (such as comint buffers).

Manager?  :)

Alex.


Here's some code that works.

(when bidi-testing
  (assert (string=
           (bidi-reorder-string
            "FOO is a word for FOOLS."
            (bidi-resolve-implicit-levels
             (bidi-resolve-weak-types
              (bidi-get-types "FOO is a word for FOOLS."))))
           "OOF is a word for SLOOF."))
  (assert (string=
           (bidi-reorder-string
            "OOF is a word for SLOOF."
            (bidi-resolve-implicit-levels
             (bidi-resolve-weak-types
              (bidi-get-types "OOF is a word for SLOOF."))))
           "FOO is a word for FOOLS.")))

The following don't work.  I'm not sure yet, but I think this cannot
be done using implicit levels only.

(string=
 (bidi-reorder-string
  "I SAW A f14 FLYING BY."
  (bidi-resolve-implicit-levels
   (bidi-resolve-weak-types
    (bidi-get-types "I SAW A f14 FLYING BY."))))
 ".YB GNIYLF f14 A WAS I")

(string=
 (bidi-reorder-string
  "he said \"I AM GOING TO london TODAY\"."
  (bidi-resolve-implicit-levels
   (bidi-resolve-weak-types
    (bidi-get-types "he said \"I AM GOING TO london TODAY\"."))))
 "he said \".YADOT london OT GNIOG MA I\"")

And here is the code for the main functions.  It gives you a kind of
overview of what has been done and what is still missing, I hope.


(defun bidi-reorder-string (str levels)
  "Reorder string STR according to resolved LEVELS.
This uses rule L2 from UAX#9.
For more information in the various rules, see the function
`bidi-apply-l1'"
  (bidi-apply-l2 str levels))


(defun bidi-resolve-weak-types (types &optional left)
  "Resolve weak bidi types in TYPES, a list of bidi categories.
TYPES is a list of bidi types to operate on.
TYPES will be modified in place.

This uses rules W1 to W7 and rules N1 and N2 from UAX#9.

The resulting list will have the same number of elements and all weak
types will be replaced by strong types.  If the optional argument LEFT
is given, the default embedding type will be right-to-left.  If LEFT is
nil, a default embedding of left-to-right will be assumed.

Since this function doesn't take care of explicit codes, all types are
considered to be at the same level.  If you decomposed a string into
stretches of differing embedding levels, you can call this function for
every such stretch.

In UAX#9 parlance, LEFT indicates the embedding level direction.  LEFT
therefore determines the text ordering type e.  We assume the
start-of-level-run type sor and the end-of-level-run type eor to be the
same as the text ordering type e.

For more information in the various rules, see the functions:
 `bidi-apply-w1'
 `bidi-apply-w2'
 `bidi-apply-w3'
 `bidi-apply-w4'
 `bidi-apply-w5'
 `bidi-apply-w6'
 `bidi-apply-w7'
 `bidi-apply-n1'
 `bidi-apply-n2'"
  (let* ((e (if left bidi-category-r bidi-category-l))
         (sor e)
         (eor e))
    (bidi-apply-n2 e
     (bidi-apply-n1 sor eor
      (bidi-apply-w7 sor
       (bidi-apply-w6
        (bidi-apply-w5
         (bidi-apply-w4
          (bidi-apply-w3
           (bidi-apply-w2 sor
            (bidi-apply-w1 sor types)))))))))))


(defun bidi-get-category (char)
  "Return category for CHAR."
  (let ((categories bidi-categories)
        (set (char-category-set char))
        result)
    (while (and (not result) categories)
      (let ((category (symbol-value (car categories))))
        (if (aref set category)
            (setq result category)
          (setq categories (cdr categories)))))
    result))

-- 
http://www.emacswiki.org/