[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Texmacs-dev] string-tokenize

From: Michael Klein
Subject: Re: [Texmacs-dev] string-tokenize
Date: Sun, 29 Jul 2007 15:32:09 +0200

Hello Henri,

I did find this function tmconcat-tokenize-math that gives me the tokens I want.

Best regards

2007/7/24, Henri Lesourd < address@hidden>:
Michael Klein wrote:

> Hello List,
> is there a procedure in scheme which tokenizes a string into
> substrings at several characters and not deleting these?
Don't know what you mean by "and not deleting these". Either you
tokenize, or either you don't (?)...

The following routine splits a string at the positions where a given
character occurs :
(define nest #t)
(define (string-tok s c)
   (define resu '())
   (define str '())
   (define nest-level 0)
   (define (traverse l)
      (if (or (null? l)
                 (or (not nest)
                     (eq? nest-level 0))
                 (eq? (car l) c))
             (set! resu (cons (list->string (reverse str)) resu))
             (set! str '())
             (if (not (null? l))
                 (traverse (cdr l)))
             (if nest (begin
                (if (eq? (car l) #\()
                    (set! nest-level (+ nest-level 1)))
                (if (eq? (car l) #\))
                    (set! nest-level (- nest-level 1)))
             (set! str (cons (car l) str))
             (traverse (cdr l))
   (set! nest-level 0)
   (traverse (string->list s))
   (reverse resu))

, and the following performs the reverse operation :
(define (string-untok l sep)
  (define s "")
  (define (traverse l)
     (if (null? l)
            (if (equal? s "")
                (set! s (car l))
                (set! s (string-append s sep (car l))))
            (traverse (cdr l))))
  (if (or (not (pair? l))
          (null? l)
      (traverse l)))

I don't know if it is exactly what you want, but you can at least
use this as a starting point.

Best, Henri

reply via email to

[Prev in Thread] Current Thread [Next in Thread]