5 Module Lexer

▼ |  | Syntax Color: Utilities

1 | Parenthesis Matching
2 | Lexer Contract and The Don’t-Stop Structure Type
3 | Racket Lexer
4 | Default Lexer
5 | Module Lexer
6 | Scribble Lexer
7 | Splay Tree for Tokenization
8 | Interface for Clients of Syntax Color Output
9 | Racket S-Expression Indentation and Navigation

On this page:

module-lexer
module-lexer*

top contents ← prev up next →

## 5 Module Lexer🔗ℹ

(require syntax-color/module-lexer)
| package: syntax-color-lib

> procedure (module-lexer in offset mode)
> | → |  |
> (or/c string? eof-object?)
> symbol?
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> exact-nonnegative-integer?
> (or/c #f
> (-> input-port? any)
> (cons/c (-> input-port? exact-nonnegative-integer?
> any/c
> any)
> any/c)
> (struct/c dont-stop
> ((cons/c (-> input-port?
> exact-nonnegative-integer?
> any/c
> any)
> any/c))))
> in : input-port?
> offset : exact-nonnegative-integer?
> | mode |  | : |  |
> (or/c #f
> (-> input-port? exact-nonnegative-integer?any)
> (cons/c (-> input-port?
> exact-nonnegative-integer?
> any/c
> any)
> any/c))

Like racket-lexer, but with several differences:

- The module-lexer function accepts an offset and lexer mode, instead of just an input port.

- In addition to the results of racket-lexer, module-lexer returns a backup distance and a new lexer mode.

- When mode is #f (indicating the start of the stream), the lexer checks in for a #lang specification.

  If a #lang line is present but the specified language does not exist, the entire in input is consumed and colored as 'error.

  If the language exists and the language provides a get-info function, then it is called with 'color-lexer. If the result is not #f, then it should be a lexer function for use with color:text<%>. The result mode is the lexer—paired with #f if the lexer is a procedure arity 3—so that future calls will dispatch to the language-supplied lexer.

  If the language is specified but it provides no get-info or 'color-lexer result, then racket-lexer is returned as the mode.

- When mode is a lexer procedure, the lexer is applied to in. The lexer’s results are returned, plus the lexer again as the mode; if the lexer produces a hash-table attribute result, however, the 'type value is extracted and returned in place of the hash table.

- When mode is a pair, then the lexer procedure in the car is applied to in, offset, and the mode in the cdr. The lexer’s results are returned, except that its mode result is paired back with the lexer procedure.

> procedure (module-lexer* in offset mode)
> | → |  |
> (or/c string? eof-object?)
> (or/c symbol?
> (and/c (hash/c symbol? any/c) immutable?))
> (or/c symbol? #f)
> (or/c number? #f)
> (or/c number? #f)
> exact-nonnegative-integer?
> (or/c #f
> (-> input-port? any)
> (cons/c (-> input-port?
> exact-nonnegative-integer?
> any/c
> any)
> any/c)
> (struct/c dont-stop
> ((cons/c (-> input-port?
> exact-nonnegative-integer?
> any/c
> any)
> any/c))))
> in : input-port?
> offset : exact-nonnegative-integer?
> | mode |  | : |  |
> (or/c #f
> (-> input-port? any)
> (cons/c (-> input-port?
> exact-nonnegative-integer?
> any/c
> any)
> any/c))

Like module-lexer, except that the attribute result propagated from a language-specific lexer can be a hash table.

Added in version 1.2 of package syntax-color-lib.

top contents ← prev up next →