Class DefaultLexer

Hierarchy (view full)

DefaultLexer
- IndentationAwareLexer

Implements

Lexer

Index

Constructors

Properties

chevrotainLexer tokenBuilder tokenTypes

Accessors

Methods

tokenize toTokenTypeDictionary

Constructors

constructor

new DefaultLexer(services): DefaultLexer
Parameters
- services: LangiumCoreServices
Returns DefaultLexer
- Defined in packages/langium/src/parser/lexer.ts:47

Properties

`Protected`chevrotainLexer

chevrotainLexer: Lexer

`Protected`tokenBuilder

tokenBuilder: TokenBuilder

`Protected`tokenTypes

tokenTypes: TokenTypeDictionary

Accessors

definition

get definition(): TokenTypeDictionary
Returns TokenTypeDictionary
Implementation of Lexer.definition
- Defined in packages/langium/src/parser/lexer.ts:61

Methods

tokenize

tokenize(text, _options?): LexerResult
Parameters
- text: string
- _options: TokenizeOptions = DEFAULT_TOKENIZE_OPTIONS
Returns LexerResult
Implementation of Lexer.tokenize
- Defined in packages/langium/src/parser/lexer.ts:65

`Protected`toTokenTypeDictionary

toTokenTypeDictionary(buildTokens): TokenTypeDictionary
Parameters
- buildTokens: TokenVocabulary
Returns TokenTypeDictionary
- Defined in packages/langium/src/parser/lexer.ts:75