TokenNavigation
methods for peek, next_token, consume, and error-handling
Instance methods
Consumes the current token if it's of the expected kind. If not, reports an error and returns nil.
Overload for specific token data
--- Identifier Consumption --- Consumes the next token as if it is an identifier. Works for variable names, type names, keywords, and operators; raises a ParseError otherwise
Consumes a newline only if it begins a new block and returns that indent. Otherwise returns nil.
Consumes a comma or newline acting as a list separator Returns the consumed token or nil if end of list Trailing comma and/or newline are consumed but not returned.
Consumes tokens until one of the target_kinds is encountered or EOF.
Returns the token that matched or the EOF token.
Consumes the current token and returns it. Advances the index. Returns the last token if already at EOF to prevent errors on multiple calls.
--- Error Reporting & Recovery ---
Consumes a newline only if it's not the end of a block
Returns true if end of block found
End of block is determined to be a newline with indent ≤ block_indent
EXCEPT if the token after newline is a closing bracket or else