class

Noir::PhpLexer

Inherits Noir::MaskedLexer < Reference < Object

PhpLexer is a hand-rolled structural lexer for PHP source. It exists to replace the per-analyzer character state machines that every PHP analyzer re-implements (balanced-brace matching, statement-end scanning, string/ comment skip ranges) with a single shared pass that is:

  • open/close-tag aware — a .php file is HTML with islands of code in it, so everything outside <?php … ?> is inert output text. Lexing it as code let a lone apostrophe in <h1>Today's report</h1> open a string that masked the rest of the file, silently dropping every route declared below it.
  • heredoc/nowdoc aware — <<<EOT … EOT / <<<'EOT' … EOT bodies are masked, so route-shaped text or stray {}; inside a heredoc can no longer leak as a false endpoint or corrupt a brace/statement bound. None of the pre-existing scanners handled <<< at all.
  • PHP-8 attribute aware — #[Route(...)] is code, not a # comment.
  • linear on multi-byte input — the source is materialised once into an Array(Char) with O(1) indexing, so CJK-commented controllers stay O(n) instead of the O(n^2) that String#[](Int) caused.

The lexer masks every non-code region (inline HTML, strings, comments, heredoc/nowdoc bodies) into spaces in @masked while preserving newlines and overall length, so the structural helpers in Noir::MaskedLexer are plain depth counters over @masked with no string-state bookkeeping of their own.

Constructors

new(source : String, *, php_mode : Bool = false)

source is a whole .php file by default, so lexing starts in HTML mode: nothing is code until the first <?php / <?=. Pass php_mode: true when source is a fragment already carved out of a PHP region (a closure body handed back for a recursive pass, say), which by construction has no opening tag of its own.

Source

Instance methods

expression_end(start_pos : Int32) : Int32

Index of the first top-level expression terminator (, ; or a closing ) ] } that would pop above the starting level) at or after start_pos. Mirrors find_arrow_expression_end.

Source
in_code?(pos : Int32) : Bool

PHP spans never nest (unlike C#/Scala interpolation), so they are sorted and disjoint: binary-search the last span starting at or before pos instead of the shared linear scan, which made a per-match in_code? walk quadratic on string-heavy files.

Source
literal?(pos : Int32) : Bool

Whether pos is inside a string literal or a heredoc/nowdoc body — blank in masked like a comment, but a value, not whitespace.

Source
source(range : Range(Int32, Int32)) : String

Source text over a character range, copied from the char array: O(range) where String#[] on multi-byte input is O(offset), which turns a loop of slices into a quadratic walk.

Source
without_comments(keep_docblocks : Bool = true) : String

The source with its comments blanked to spaces and everything else — string literals, heredocs, inline HTML — left as written. Same length and line breaks as the source, so offsets and line numbers computed on it hold for the original. This is what lets a regex-driven adapter skip // $app->get('/old') without losing the route text inside strings.

Docblocks (/** … */) are kept unless keep_docblocks is false: Doctrine-style @Route annotations live in them, and PHP only reads annotations from that form.

Source