Analyzer

Detector reader → worker content channel. Each buffered entry holds a full file's content (up to the size budget the media filter applied — MAX_FILE_SIZE, or MAX_SPEC_FILE_SIZE for specification documents), so this bounds worst-case unreclaimable memory as capacity * that budget.

Was raised 16 -> 64 under the theory that the single reader fiber was blocking on send and stalling disk I/O. Re-measured (#2362) on a heavy-load corpus (multi-detector + passive-scan + -T + --ai-context, ~80MB of large files): 16 and 64 perform identically (~14.7s avg either way over 9 interleaved runs). There is no preview_mt build, so reader and workers are cooperative fibers on one thread with no blocking-I/O yield point to recover — a deeper buffer just lets the reader run further ahead without unblocking anything. Kept at 16.

URL for a file-path-routed stack — plain PHP, JSP, CFML, Classic ASP, WebForms — where the file's location under the web root is the route.

The root is the configured scan base unless one of markers names a document root deeper in the tree. The last occurrence wins: for app/wwwroot/x/htdocs/y.asp the deepest marker is the real root, whereas testing markers in list order would pick whichever name happened to be listed first.

Keep markers unambiguous. Generic names like public/ or www/ collide with build output (docs/public/) — the false positive FileHelper#get_public_files documents. The file's location relative to the configured scan base, /-separated and rooted with a leading /.

Convention filters — "is this file under includes/?", "is this a test fixture?" — must be applied to this, never to the absolute path. A checkout that merely lives under a directory with a matching name otherwise matches every file in it: scanning the same Classic ASP site from /srv/inc/site instead of /srv/site dropped 33 of its 35 endpoints, because every page looked like an #include fragment.

Callees feed --include-callee (direct output) and --ai-context (aggregated review context). Analyzers should consult this before running their callee extractor so the work is skipped on default scans where neither flag is set.

Whether content matches markers, skipping PCRE2's per-call UTF-8 revalidation. Analyzer content always comes from read_file_content (or the detector cache it reads through), so the subject is known-valid UTF-8 — see Noir::TextFile::MATCH_OPTIONS.

Use this in place of a chain of String#includes? over whole file content: String#includes? runs Rabin-Karp per marker, while one precompiled Regex.union of the same literals is the same predicate in a single JIT-compiled pass. A single includes? on a short string (one source line, a path segment) is not worth converting — the win scales with the size of the subject.

HTTP_X_FORWARDED_FOR -> x-forwarded-for.

CGI-style server-variable collections (Request.ServerVariables in ASP/WebForms, cgi.* in CFML) expose inbound headers under an HTTP_-prefixed, underscore-separated name. Everything without that prefix (REMOTE_ADDR, SCRIPT_NAME, ...) is server state, not a request parameter, and returns nil.

line_at_offset for a byte offset — what Regex::MatchData#byte_begin and the byte-wise scanners hand back. Counting newline bytes over a slice is both correct for multibyte content (a UTF-8 continuation byte never collides with an ASCII value) and free of the per-call string allocation the char form pays.

1-based line the character at offset sits on.

The content[0...offset].count('\n') + 1 this names is open-coded in a couple of dozen analyzers; naming it keeps the off-by-one in one place (a 0 offset is line 1, not line 0 — PathInfo drops a 0 as "no line"). Callers that need a line for many offsets in the same file should build their own newline table instead: this is linear in offset.

1-based line number for a character offset into content.

Counts newlines over the raw byte buffer rather than slicing content[0...index], which would be an O(n) copy per call. The scan is correct because \n is ASCII and never appears inside a UTF-8 multi-byte sequence.

Takes a CHAR offset. Feeding it a byte offset silently reports a line that is too low on non-ASCII input — the defect four JS analyzers carried until they were folded in here.

Preferred overload: accepts a file list and creates both the producer and worker fibers inside a single WaitGroup so every fiber is tracked. The bare-spawn producer in the channel-based overload below was an orphan that could trigger "can't resume a running fiber" under Crystal ≥1.20's M:N scheduler when multiple analyzers ran concurrently.

Prefer the detector-populated cache over a fresh disk read. On cache miss (budget exhausted, cache cleared between runs, path not registered via register_file) falls back to a disk read.

Analyzers migrating from direct File.read(path, ...) calls should use this helper so the second read of files the detector already loaded is free.

Overridden by analyzer_for in every registered analyzer. The base value exists so shared code can call it unconditionally; it is deliberately NOT a self.tech_name fallback, which would turn "forgot analyzer_for" from a compile error into an analyzer that silently never runs.

Order-preserving dedup of params by (name, param_type). Analyzers that accumulate every reference they see (rather than every route) collect the same key many times over.

Declares which tech this analyzer implements.

class Gin < GoEngine
  analyzer_for "go_gin"
end

initialize_analyzers reads the registry off the classes themselves, so this is the only place the name is written. Before it, the mapping lived in a hand-maintained define_analyzers list — a second place to remember, where forgetting produced no error and no failing spec, just an analyzer that never ran. Mirrors Detector.detector_for.

Every concrete class under the Analyzer:: namespace must declare one: the derivation calls tech_name on each, so omitting it is a compile error rather than a silent no-op. Shared base classes (GoEngine, SpecificationEngine, …) are abstract and sit outside that set.