class

Glimmer::Highlight::SimpleRegexLexer

Inherits Glimmer::Highlight::Lexer < Reference < Object

SimpleRegexLexer provides a basic regex-based tokenization.

Languages can define patterns to match different token types.

Constructors

new(language_name : String, patterns : Array(Pattern), language_aliases : Array(String) = [] of String)

Instance methods

aliases

Returns a list of aliases for this language (e.g., "js" for "javascript").

language

Returns the name of the language this lexer handles.

language_aliases

language_aliases=(language_aliases : Array(String))

language_name

language_name=(language_name : String)

patterns

patterns=(patterns : Array(Pattern))

tokenize(source : String) : Array(Glimmer::Highlight::Token)

Tokenizes the given source code into an array of tokens.

Nested types

Glimmer::Highlight::SimpleRegexLexer::Pattern