libunibreak: linebreak.c File Reference

libunibreak 7.0

Loading...
Searching...
No Matches
Macros | Enumerations | Functions
linebreak.c File Reference

Implementation of the line breaking algorithm as described in Unicode Standard Annex 14. More...

`#include <assert.h>`
`#include <stddef.h>`
`#include <stdint.h>`
`#include <string.h>`
`#include "eastasianwidthdef.h"`
`#include "linebreak.h"`
`#include "linebreakdef.h"`
Include dependency graph for linebreak.c:

## Macros
#define | UB_LB25_OPT_HACK 1
#define | LINEBREAK_UNDEFINED -1
| Special value used internally to indicate an undefined break result.
#define | INVALID_POS ((size_t)-1)
| Special value used internally to mark an invalid position.
#define | ENDS_WITH(str, suffix)

## Enumerations
enum | BreakAction { DIR_BRK , IND_BRK , CMI_BRK , CMP_BRK , PRH_BRK }
| Enumeration of break actions. More...

## Functions
void | init_linebreak (void)
| Does nothing.
void | lb_init_break_context (struct LineBreakContext *lbpCtx, utf32_t ch, const char *lang)
| Initializes line breaking context for a given language.
int | lb_process_next_char (struct LineBreakContext *lbpCtx, utf32_t ch)
| Updates LineBreakingContext for the next codepoint and returns the detected break.
enum LineBreakClass | lb_get_char_class (const struct LineBreakContext *lbpCtx, utf32_t ch)
| Gets the line breaking class of a character for a line breaking context.
size_t | set_linebreaks (const void *s, size_t len, const char *lang, enum BreakOutputType outputType, char *brks, get_next_char_t get_next_char)
| Sets the line breaking information for a generic input string.
void | set_linebreaks_utf8 (const utf8_t *s, size_t len, const char *lang, char *brks)
| Sets the line breaking information for a UTF-8 input string.
size_t | set_linebreaks_utf8_per_code_point (const utf8_t *s, size_t len, const char *lang, char *brks)
| Sets the line breaking information for a UTF-8 input string.
void | set_linebreaks_utf16 (const utf16_t *s, size_t len, const char *lang, char *brks)
| Sets the line breaking information for a UTF-16 input string.
size_t | set_linebreaks_utf16_per_code_point (const utf16_t *s, size_t len, const char *lang, char *brks)
| Sets the line breaking information for a UTF-16 input string.
void | set_linebreaks_utf32 (const utf32_t *s, size_t len, const char *lang, char *brks)
| Sets the line breaking information for a UTF-32 input string.
int | is_line_breakable (utf32_t char1, utf32_t char2, const char *lang)
| Tells whether a line break can occur between two Unicode characters.

## Detailed Description

Implementation of the line breaking algorithm as described in Unicode Standard Annex 14.

Author
  Wu Yongwei
  Petr Filipsky

## Macro Definition Documentation

## ◆ ENDS_WITH

#define ENDS_WITH | ( |  | str,
|  |  | suffix )

Value:
ends_with((str), (suffix), sizeof(suffix) - 1)

## ◆ INVALID_POS

#define INVALID_POS ((size_t)-1)

Special value used internally to mark an invalid position.

## ◆ LINEBREAK_UNDEFINED

#define LINEBREAK_UNDEFINED -1

Special value used internally to indicate an undefined break result.

## ◆ UB_LB25_OPT_HACK

#define UB_LB25_OPT_HACK 1

## Enumeration Type Documentation

## ◆ BreakAction

enum BreakAction

Enumeration of break actions.

They are used in the break action pair table #baTable.

Enumerator
DIR_BRK | Direct break opportunity.
IND_BRK | Indirect break opportunity.
CMI_BRK | Indirect break opportunity for combining marks.
CMP_BRK | Prohibited break for combining marks.
PRH_BRK | Prohibited break.

## Function Documentation

## ◆ init_linebreak()

void init_linebreak | ( | void |  | ) |

Does nothing.

This is kept for binary compatibility.

## ◆ is_line_breakable()

int is_line_breakable | ( | utf32_t | char1,
|  | utf32_t | char2,
|  | const char * | lang )

Tells whether a line break can occur between two Unicode characters.

This is a wrapper function to expose a simple interface. Generally speaking, it is better to use set_linebreaks_utf32 instead, since complicated cases involving combining marks, spaces, etc. cannot be correctly processed.

Parameters

  char1 | the first Unicode character
  char2 | the second Unicode character
  lang | language of the input

Returns
  one of LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR

## ◆ lb_get_char_class()

enum LineBreakClass lb_get_char_class | ( | const struct LineBreakContext * | lbpCtx,
|  | utf32_t | ch )

Gets the line breaking class of a character for a line breaking context.

This function will check the language-specific data first, and then the default data if there is no language-specific property available for the character.

Parameters

  lbpCtx | pointer to the line breaking context
  ch | character to check

Returns
  the line breaking class if found; `LBP_XX` otherwise

## ◆ lb_init_break_context()

void lb_init_break_context | ( | struct LineBreakContext * | lbpCtx,
|  | utf32_t | ch,
|  | const char * | lang )

Initializes line breaking context for a given language.

Parameters

  [in,out] | lbpCtx | pointer to the line breaking context
  [in] | ch | the first character to process
  [in] | lang | language of the input

Postcondition
  the line breaking context is initialized

## ◆ lb_process_next_char()

int lb_process_next_char | ( | struct LineBreakContext * | lbpCtx,
|  | utf32_t | ch )

Updates LineBreakingContext for the next codepoint and returns the detected break.

This function is deprecated, as it cannot support fixups, as required by LB25 tailoring (and some more recent rules). See the implementation of set_linebreaks for the fixup logic.

Parameters

  [in,out] | lbpCtx | pointer to the line breaking context
  [in] | ch | Unicode codepoint

Returns
  break result, one of LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, and LINEBREAK_NOBREAK

Postcondition
  the line breaking context is updated

## ◆ set_linebreaks()

size_t set_linebreaks | ( | const void * | s,
|  | size_t | len,
|  | const char * | lang,
|  | enum BreakOutputType | outputType,
|  | char * | brks,
|  | get_next_char_t | get_next_char )

Sets the line breaking information for a generic input string.

Currently, this implementation has customization for the following ISO 639-1 language codes (for lang):

- de (German)
- en (English)
- es (Spanish)
- fr (French)
- ja (Japanese)
- ko (Korean)
- ru (Russian)
- zh (Chinese)

In addition, a suffix `"-strict"` may be added to indicate strict (as versus normal) line-breaking behaviour. See the Conditional Japanese Starter section of UAX #14 for more details.

Parameters

  [in] | s | input string
  [in] | len | length of the input
  [in] | lang | language of the input
  [in] | outputType | output per code-unit or per code-point
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR
  [in] | get_next_char | function to get the next UTF-32 character

Returns
  The number of entries in brks filled. This is equal to the number of code-points or code-units in the source string, depending on the outputType parameter.

## ◆ set_linebreaks_utf16()

void set_linebreaks_utf16 | ( | const utf16_t * | s,
|  | size_t | len,
|  | const char * | lang,
|  | char * | brks )

Sets the line breaking information for a UTF-16 input string.

Parameters

  [in] | s | input UTF-16 string
  [in] | len | length of the input
  [in] | lang | language of the input
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR

See also
  set_linebreaks for a note about lang.

## ◆ set_linebreaks_utf16_per_code_point()

size_t set_linebreaks_utf16_per_code_point | ( | const utf16_t * | s,
|  | size_t | len,
|  | const char * | lang,
|  | char * | brks )

Sets the line breaking information for a UTF-16 input string.

Parameters

  [in] | s | input UTF-16 string
  [in] | len | length of the input
  [in] | lang | language of the input
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK

Returns
  The number of entries in brks filled. This is equal to the number of code-points in the source string.

See also
  set_linebreaks for a note about lang.

## ◆ set_linebreaks_utf32()

void set_linebreaks_utf32 | ( | const utf32_t * | s,
|  | size_t | len,
|  | const char * | lang,
|  | char * | brks )

Sets the line breaking information for a UTF-32 input string.

Parameters

  [in] | s | input UTF-32 string
  [in] | len | length of the input
  [in] | lang | language of the input
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR

See also
  set_linebreaks for a note about lang.

## ◆ set_linebreaks_utf8()

void set_linebreaks_utf8 | ( | const utf8_t * | s,
|  | size_t | len,
|  | const char * | lang,
|  | char * | brks )

Sets the line breaking information for a UTF-8 input string.

Parameters

  [in] | s | input UTF-8 string
  [in] | len | length of the input
  [in] | lang | language of the input
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR

See also
  set_linebreaks for a note about lang.

## ◆ set_linebreaks_utf8_per_code_point()

size_t set_linebreaks_utf8_per_code_point | ( | const utf8_t * | s,
|  | size_t | len,
|  | const char * | lang,
|  | char * | brks )

Sets the line breaking information for a UTF-8 input string.

Parameters

  [in] | s | input UTF-8 string
  [in] | len | length of the input
  [in] | lang | language of the input
  [out] | brks | pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK

Returns
  The number of entries in brks filled. This is equal to the number of code-points in the source string.

See also
  set_linebreaks for a note about lang.