Class DefaultLexer

Hierarchy (View Summary)

DefaultLexer
- IndentationAwareLexer

Implements

Lexer

Index

Constructors

Properties

chevrotainLexer errorMessageProvider tokenBuilder tokenTypes

Accessors

Methods

tokenize toTokenTypeDictionary

Constructors

constructor

new DefaultLexer(services: LangiumCoreServices): DefaultLexer
Parameters
- services: LangiumCoreServices
Returns DefaultLexer
- Defined in packages/langium/src/parser/lexer.ts:59

Properties

`Protected`chevrotainLexer

chevrotainLexer: Lexer

`Protected` `Readonly`errorMessageProvider

errorMessageProvider: ILexerErrorMessageProvider

`Protected` `Readonly`tokenBuilder

tokenBuilder: TokenBuilder

`Protected`tokenTypes

tokenTypes: TokenTypeDictionary

Accessors

definition

get definition(): TokenTypeDictionary
Returns TokenTypeDictionary
Implementation of Lexer.definition
- Defined in packages/langium/src/parser/lexer.ts:75

Methods

tokenize

tokenize(text: string, _options?: TokenizeOptions): LexerResult
Parameters
- text: string
- _options: TokenizeOptions = DEFAULT_TOKENIZE_OPTIONS
Returns LexerResult
Implementation of Lexer.tokenize
- Defined in packages/langium/src/parser/lexer.ts:79

`Protected`toTokenTypeDictionary

toTokenTypeDictionary(buildTokens: TokenVocabulary): TokenTypeDictionary
Parameters
- buildTokens: TokenVocabulary
Returns TokenTypeDictionary
- Defined in packages/langium/src/parser/lexer.ts:89