[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-gnulib] addition: linebreak.h, linebreak.c
From: |
Bruno Haible |
Subject: |
[Bug-gnulib] addition: linebreak.h, linebreak.c |
Date: |
Thu, 3 Apr 2003 21:05:03 +0200 (CEST) |
Hi,
I propose to add the internationalized line breaking engine from GNU
gettext to gnulib. Unlike the "break at spaces" heuristic which
doesn't work for Chinese text, this code - an implementation of the
Unicode line breaking algorithm - works for most languages of the
world. (Still a south-east Asian language is not supported for which
I'd need a free dictionary.)
I want to put it into gnulib in the hope that it will be used e.g. as
an alternative algorithm (option -U, --unicode) of 'fmt' in coreutils.
The file defines the functions in 3 variants, for UTF-8, UTF-16 and
UCS-4 strings; I usually comment out two of them through "#if 0"
in order to reduce the size of the generated executables.
Furthermore a fourth variant, which works on multibyte strings
(implemented via iconv on top of the UTF-8 code). It is the latter
which most programs use.
Speak up if you have objections!
Bruno
========================== linebreak.h ==========================
/* linebreak.h - line breaking of Unicode strings
Copyright (C) 2001-2002 Free Software Foundation, Inc.
Written by Bruno Haible <address@hidden>, 2001.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */
#ifndef _LINEBREAK_H
#define _LINEBREAK_H
/* Get size_t. */
#include <stddef.h>
/* Display width. */
/* These functions are locale dependent. The encoding argument identifies
the encoding (e.g. "ISO-8859-2" for Polish). */
/* Return the encoding of the current locale. */
extern const char * locale_charset (void);
/* Determine number of column positions required for UC. */
extern int uc_width (unsigned int uc, const char *encoding);
/* Determine number of column positions required for first N units
(or fewer if S ends before this) in S. */
extern int u8_width (const unsigned char *s, size_t n, const char *encoding);
extern int u16_width (const unsigned short *s, size_t n, const char *encoding);
extern int u32_width (const unsigned int *s, size_t n, const char *encoding);
/* Line breaking. */
enum {
UC_BREAK_UNDEFINED,
UC_BREAK_PROHIBITED,
UC_BREAK_POSSIBLE,
UC_BREAK_MANDATORY,
UC_BREAK_HYPHENATION
};
/* Determine the line break points in S, and store the result at p[0..n-1].
p[i] = UC_BREAK_MANDATORY means that s[i] is a line break character.
p[i] = UC_BREAK_POSSIBLE means that a line break may be inserted between
s[i-1] and s[i].
p[i] = UC_BREAK_HYPHENATION means that a hyphen and a line break may be
inserted between s[i-1] and s[i]. But beware of language dependent
hyphenation rules.
p[i] = UC_BREAK_PROHIBITED means that s[i-1] and s[i] must not be separated.
*/
extern void u8_possible_linebreaks (const unsigned char *s, size_t n, const
char *encoding, char *p);
extern void u16_possible_linebreaks (const unsigned short *s, size_t n, const
char *encoding, char *p);
extern void u32_possible_linebreaks (const unsigned int *s, size_t n, const
char *encoding, char *p);
extern void mbs_possible_linebreaks (const char *s, size_t n, const char
*encoding, char *p);
/* Choose the best line breaks, assuming the uc_width function.
Return the column after the end of the string.
o is an optional override; if o[i] != UC_BREAK_UNDEFINED, o[i] takes
precedence over p[i] as returned by the *_possible_linebreaks function.
*/
extern int u8_width_linebreaks (const unsigned char *s, size_t n, int width,
int start_column, int at_end_columns, const char *o, const char *encoding, char
*p);
extern int u16_width_linebreaks (const unsigned short *s, size_t n, int width,
int start_column, int at_end_columns, const char *o, const char *encoding, char
*p);
extern int u32_width_linebreaks (const unsigned int *s, size_t n, int width,
int start_column, int at_end_columns, const char *o, const char *encoding, char
*p);
extern int mbs_width_linebreaks (const char *s, size_t n, int width, int
start_column, int at_end_columns, const char *o, const char *encoding, char *p);
#endif /* _LINEBREAK_H */
========================== linebreak.c ==========================
too large for this mail
- [Bug-gnulib] addition: linebreak.h, linebreak.c,
Bruno Haible <=