Interface Lexer

interface Lexer {
definition: TokenTypeDictionary;
tokenize(text: string, options?: TokenizeOptions): LexerResult;
}

Implemented by

DefaultLexer

Index

Properties

Methods

Properties

`Readonly`definition

definition: TokenTypeDictionary

Methods

tokenize

tokenize(text, options?): LexerResult
Parameters
- text: string
- Optionaloptions: TokenizeOptions
Returns LexerResult
- Defined in packages/langium/src/parser/lexer.ts:49