class

Noir::PhpLexer

Inherits Noir::MaskedLexer < Reference < Object

PhpLexer is a hand-rolled structural lexer for PHP source. It exists to replace the per-analyzer character state machines that every PHP analyzer re-implements (balanced-brace matching, statement-end scanning, string/ comment skip ranges) with a single shared pass that is:

  • open/close-tag aware — a .php file is HTML with islands of code in it, so everything outside <?php … ?> is inert output text. Lexing it as code let a lone apostrophe in <h1>Today's report</h1> open a string that masked the rest of the file, silently dropping every route declared below it.
  • heredoc/nowdoc aware — <<<EOT … EOT / <<<'EOT' … EOT bodies are masked, so route-shaped text or stray {}; inside a heredoc can no longer leak as a false endpoint or corrupt a brace/statement bound. None of the pre-existing scanners handled <<< at all.
  • PHP-8 attribute aware — #[Route(...)] is code, not a # comment.
  • linear on multi-byte input — the source is materialised once into an Array(Char) with O(1) indexing, so CJK-commented controllers stay O(n) instead of the O(n^2) that String#[](Int) caused.

The lexer masks every non-code region (inline HTML, strings, comments, heredoc/nowdoc bodies) into spaces in @masked while preserving newlines and overall length, so the structural helpers in Noir::MaskedLexer are plain depth counters over @masked with no string-state bookkeeping of their own.

Constructors

new(source : String, *, php_mode : Bool = false)

source is a whole .php file by default, so lexing starts in HTML mode: nothing is code until the first <?php / <?=. Pass php_mode: true when source is a fragment already carved out of a PHP region (a closure body handed back for a recursive pass, say), which by construction has no opening tag of its own.

Source

Instance methods

expression_end(start_pos : Int32) : Int32

Index of the first top-level expression terminator (, ; or a closing ) ] } that would pop above the starting level) at or after start_pos. Mirrors find_arrow_expression_end.

Source
tokens

Lazily produce a flat token stream over the source: structural delimiters, ->/::/=> operators, identifiers, $variables, and one token per string/comment/heredoc span. This is the reusable miniparser surface for consumers that want to walk PHP structurally (e.g. following a Route::a(...)->b(...)->group(...) method chain).

Source