class

Obelisk::Lexers::Python

Inherits Obelisk::RegexLexer < Obelisk::Lexer < Reference < Object

Python language lexer Optimized with regex constants and consolidated string handling

Constants

ARROW = /->/
B_PREFIX = /[bB]/
BUILTINS = /\b(?:abs|all|any|ascii|bin|bool|bytes|callable|chr|classmethod|compile|complex|delattr|dict|dir|divmod|enumerate|eval|filter|float|format|frozenset|getattr|globals|hash|help|hex|id|input|int|isinstance|issubclass|iter|len|list|locals|map|max|min|next|object|oct|open|ord|pow|print|range|repr|reversed|round|set|setattr|slice|sorted|staticmethod|str|sum|super|tuple|type|vars|zip)\b/

Built-in functions (simplified from ~80 to most common)

CLASS_NAME = /\b[A-Z][a-zA-Z0-9_]*\b/

Class names (CapitalizedWords)

DECORATOR = /@[\w.]+/

Decorators

DOUBLE_QUOTE = /"/

String delimiters

ESCAPE_ANY = /\\./
ESCAPE_HEX = /\\x[0-9a-fA-F]{2}/
ESCAPE_OCTAL = /\\[0-7]{1,3}/
ESCAPE_SIMPLE = /\\[\\\"'nrtbfav]/

Escape sequences (consolidated for all string types)

ESCAPE_UNICODE = /\\u[0-9a-fA-F]{4}/
ESCAPE_UNICODE_LONG = /\\U[0-9a-fA-F]{8}/
ESCAPE_UNICODE_NAME = /\\N\{[^}]+\}/
EXCEPTIONS = /\b(?:BaseException|Exception|ArithmeticError|AssertionError|AttributeError|EOFError|FloatingPointError|GeneratorExit|ImportError|IndentationError|IndexError|KeyError|KeyboardInterrupt|MemoryError|ModuleNotFoundError|NameError|NotImplementedError|OSError|OverflowError|RecursionError|ReferenceError|RuntimeError|StopIteration|StopAsyncIteration|SyntaxError|SystemError|TypeError|UnboundLocalError|ValueError|ZeroDivisionError|IOError)\b/

Exceptions (simplified)

F_PREFIX = /[fF]/

String prefixes

FUNCTION_CALL = /[a-zA-Z_]\w*(?=\s*\()/

Function call pattern (lookahead)

IDENTIFIER = /[a-zA-Z_]\w*/

Identifiers

INTERPOLATION_END = /\}/
INTERPOLATION_START = /\{/

F-string interpolation

KEYWORDS = /\b(?:False|None|True|and|as|assert|async|await|break|class|continue|def|del|elif|else|except|finally|for|from|global|if|import|in|is|lambda|nonlocal|not|or|pass|raise|return|try|while|with|yield)\b/

Keywords

LINE_COMMENT = /#.*?(?=\n|$)/
MAGIC_METHODS = /__(?:init|new|del|repr|str|eq|ne|lt|le|gt|ge|hash|bool|getattr|setattr|iter|len|getitem|setitem|delitem|call|add|sub|mul|truediv|floordiv|mod|pow|neg|pos|abs|invert|int|float|enter|exit|await|aenter|aexit|contains|format|bytes|hash|size|length|getitem|setitem|missing|iter|next|bool|nonzero|index|getitem|setitem|delitem|init|call|reduce|reduce_ex|copy|deepcopy|cmp|rcmp|eq|ne|lt|le|gt|ge|cmp|hash|nonzero|unicode|trunc|floor|ceil|floor|ceil|round|trunc|radd|rsub|rmul|rtruediv|rfloordiv|rmod|rdivmod|rpow|rlshift|rrshift|rand|rxor|ror|iadd|isub|imul|itruediv|ifloordiv|imod|ipow|ilshift|irshift|iand|ixor|ior|concat|repeat|inplace_add|inplace_sub|inplace_multiply|inplace_mod|inplace_truediv|inplace_floordiv|inplace_pow|inplace_lshift|inplace_rshift|inplace_and|inplace_xor|inplace_or|native|getattr_handle|getattribute|setattr|delattr)__/

Magic methods (simplified to most common)

NUMBER_BIN = /0[bB][01]+(_[01]+)*/

Numbers

NUMBER_COMPLEX = /[jJ]/

Complex numbers

NUMBER_EXP = /\d+[eE][+-]?\d+/
NUMBER_FLOAT = /\d+\.\d*([eE][+-]?\d+)?/
NUMBER_FLOAT_LEADING = /\.\d+([eE][+-]?\d+)?/
NUMBER_HEX = /0[xX][0-9a-fA-F]+(_[0-9a-fA-F]+)*/
NUMBER_INT = /\d+(_\d+)*/
NUMBER_OCT = /0[oO][0-7]+(_[0-7]+)*/
OPERATORS = /[+\-*\/%<>=!&|^~]+|\/\/|\*\*|<<|>>|<=|>=|==|!=/

Operators (ordered: longer patterns first)

PUNCTUATION = /[.,;:()\[\]{}|]/

Punctuation

R_PREFIX = /[rR]/
SHEBANG = /^#!/
SINGLE_QUOTE = /'/
STRING_DOUBLE_CONTENT = /[^"\\]+/

String content patterns

STRING_DOUBLE_CONTENT_NO_BRACE = /[^"\\{]+/
STRING_SINGLE_CONTENT = /[^'\\]+/
STRING_SINGLE_CONTENT_NO_BRACE = /[^'\\{]+/
TRIPLE_DOUBLE = /"""/
TRIPLE_SINGLE = /'''/
WHITESPACE = /\s+/

Whitespace and comments

Instance methods

analyze(text : String) : Float32

Analyze text to determine if this lexer can handle it Returns a score from 0.0 to 1.0

Source
config
Source
rules
Source