Noir::JSRouteExtractor
JSRouteExtractor provides a unified interface for extracting routes from JavaScript files
Constants
Pre-filter for extract_routes: returns false when content
contains no shape the JS parser knows how to emit (any verb
invocation pattern like .get(/.post(/... or Fastify/Restify
.route(, plus Express-style mounts .use( which feed into the
cross-file router prefix table). Matching is millions of times
cheaper than tokenizing the file.
Client-side UI framework imports. A file that imports a browser
UI framework (Vue, React, Angular, Svelte, Solid, Preact) and its
satellite libs (pinia, vue-router, @vueuse, react-router, ...) is
SPA/frontend code, not an HTTP server. Its route-shaped calls are
outbound API-client requests against a configured client — e.g.
directus's admin app does api.get(/users/${userId}) where api
is a wrapped axios instance imported from @/api. The existing
axios/got/ky markers miss these because the wrapper hides the raw
client behind a local module, but the UI-framework import is an
unambiguous "this is browser code" signal. directus's admin SPA
alone parks ~61 phantom Express endpoints across
app/src/{stores,composables,layouts,...} this way. Like the
test-stub markers, this is gated by the HTTP-server-import
exemption below: an SSR entrypoint that imports BOTH vue and
express keeps its routes.
A @fastify/autoload plugin module names no framework at all — it
receives the instance as a parameter:
export default async function (fastify) {
fastify.get('/status', handler)
}
so no import marker fires and the shared extractor happily reads those
registrations for whichever framework asked. Express reported /status
and /go out of the fastify autoload fixture, without the autoPrefix
the Fastify analyzer applies. The receiver name is the only evidence in
the file, and fastify is unambiguous — nothing else calls its app
instance that. Consulted only when the file imports no HTTP server.
Constructors that build an HTTP client. client.get('/todo', cb) is
the same call shape as a route registration, so a client module reads as
a server to the shared extractor: the Express analyzer reported
GET /todo and DELETE /todo/example out of a restify-clients module
that only calls a remote API.
The restify analyzer already refused these; the check belongs here so
every framework sharing the extractor gets it. It is deliberately
limited to client constructors rather than client package names —
a genuine route file may well require('axios') to call downstream
services, and gating on that would drop its routes.
Real HTTP-server library imports. When any of these is present alongside a test-stub marker, the file is doing legitimate server work (e.g., spinning up a test instance of an Express app) and we still want to extract its routes.
Average bytes-per-line above which a file is considered dominated by long lines, i.e. a bundle rather than hand-written source that merely carries one fat literal (a big inline JSON seed, an embedded base64 data URI, a long regex). Real code keeps the average low because it has many short lines around any such literal.
Byte length above which a single source line is considered "long".
Hand-written JS/TS keeps lines well under this even in dense route
tables (noir's own widest fixture line is ~150 bytes); webpack/
rollup/esbuild bundles and *.min.js assets routinely pack tens of
thousands of bytes onto one line, so 5000 leaves a wide margin.
NB: the metric is bytes, not characters — a dense single-line
non-Latin blob (>=5000 bytes but fewer chars) can trip it, which is
acceptable since real route registrations are ASCII verbs/paths.
Sibling JS/TS server frameworks that DON'T call extract_routes
(NestJS uses decorators, Hapi/Elysia/AdonisJS have their own
tree-sitter extractors) but whose files are still walked by every
JS/TS analyzer's parallel_file_scan. These markers are exclusion-
only: recognizing "this file belongs to NestJS" keeps Express/
Fastify/Koa/Hono/Restify from re-extracting a route-shaped call
(an inline example, a raw app.use() bridge, ...) out of it.
True when content carries definitive import evidence of a
different shared-extractor (or sibling) framework than
framework, with no evidence of framework itself. Guards the main
extract_routes call in Express/Fastify/Koa/Hono/Restify's
analyzers: a file that only imports 'hono' should never be
re-attributed to js_express just because its .get()/.post()
chaining looks the same as Express's (issue #2368) — whichever
analyzer runs over it first no longer matters once #2367 made the
dedup tiebreak deterministic, because the over-matching analyzer
never produces a competing endpoint in the first place.
Files with no shared-extractor/sibling import at all (a router
module that only receives app/router as a bare parameter, with
no import in that file) return false here and fall through to the
existing, permissive whole-tree scan — there's no import to
disambiguate on, so narrowing further is left as follow-up scope
(a confirmed package.json dependency or cross-file mount signal
would be needed to resolve those).
Per-framework precompiled unions of the two tables above: the
framework's own import markers, and the markers of every other
shared-extractor framework. Together with the sibling union this
turns the up-to-44-literal includes? walk below into three
matches. Five analyzers call this on every JS/TS file, so the old
shape scanned some trees over 200 times per file.
Import markers for the five frameworks whose analyzers call
extract_routes directly and therefore share its framework-agnostic
verb-chaining shape (.get(/.post(/...). Keyed by the same Symbol
each analyzer passes to other_shared_extractor_framework? below.
Extract static path declarations from JavaScript content
Returns array of hashes with static_path (URL prefix) and file_path (directory)
framework scopes the scan to one framework's static-mount idiom so a
framework analyzer running over a sibling project's file (every JS
analyzer walks all .js/.ts files) doesn't pick up another
framework's static declaration and re-emit it under the wrong tech.
nil runs every pattern (back-compat for un-scoped callers).
True when the file's route-shaped calls are almost certainly mock-server stubs (Ember pretender, MSW, nock, ...) rather than real route registrations. Two routes:
Path markers strict enough that the HTTP-server-import
exemption shouldn't apply: /e2e/, /cypress/, /playwright/,
/__mocks__/, /__tests__/, /e2e-tests/, /mirage/. Real
apps never park production handlers under any of these — even
when the harness file imports express to spin up a faked
service (Ghost's e2e/helpers/services/stripe/fake-stripe-server.ts
is the canonical example). Keeping the exemption out of these
paths catches the harness fakes without affecting legit
backend code.
- Filename markers fire unconditionally —
foo.test.tsis a test no matter what it imports. - Strict path markers also fire unconditionally —
e2e/,cypress/, etc. are dedicated test/mock trees that never contain production handlers, even when the harness file imports a server lib. - Library + the remaining directory markers honor an
exemption — if the file also imports a real HTTP server
lib (express, fastify, ...), keep it so legit test-server
harnesses (e.g. mattermost's
webhook_serve.js) keep their routes.include_client_frameworkscontrols whether a client-side UI framework import (Vue/React/...) counts as a skip signal. It must be ON for the verb-DSL extractor (a React/Vue file callingapi.get(...)is an outbound client call, not a route), but OFF for analyzers whose OWN route definitions live in client-side files — TanStack Router (createFileRoute) and tRPC route modules routinelyimport { ... } from 'react', and skipping them on that basis dropped every such route. The test-stub library markers (msw/supertest/...) and path/ filename markers still apply in both modes. Precompiled unions of the marker lists above.Regex.unionescapes every String argument, so each is exactly theany? includes?it replaces — but the content lists are long (72 test-stub libraries, 32 client frameworks, 26 server libraries), and every JS/TS file in the tree used to be walked once per literal.
Hard test-file markers: when the filename itself follows a
ubiquitous test convention, the file practically never
defines real routes. Skip these even when the file imports
a real HTTP server lib — NestJS e2e tests routinely import
@nestjs/platform-express for type-only references, and
supertest harnesses import the same modules they exercise.
The supertest request(app).get(...) shape would otherwise
ride the HTTP-server-import exemption straight back into the
parser.
Test-fixture libraries whose API mimics route registration:
pretender/miragejs expose server.get("/x", ...), MSW and
nock expose handler builders, sinon-via-faker likewise. When
these libraries are imported, virtually every route-shaped call
in the file is a stub, not a real registration. Substring match
is enough — these tokens never appear in production HTTP server
source under normal circumstances.
Path-level evidence that a file is a mock-server fixture.
Pretender helpers in particular get a helper/this arg and
call this.get(...) / this.post(...) directly, so they have
no library-name imports the content filter can hook on — fall
back to the convention-based filename match.
Class methods
Delegate to JSLiteralScanner for literal-aware brace matching
Delegate to JSLiteralScanner for literal-aware paren matching
1-based line number for a CHAR index into content.
True when content looks like a minified/bundled asset rather than
hand-written source. Two conditions must BOTH hold so we never drop
the routes of a normal file that just happens to carry one long
line (issue #1903 review):
- at least one line reaches MINIFIED_LINE_THRESHOLD bytes, and
- the rest of the file — every code line except that single fattest one — still averages MINIFIED_AVG_LINE_THRESHOLD bytes or more, i.e. long lines dominate and newline density is low.
Condition (2) deliberately sets aside the longest line and every
blank/comment-only line before averaging. Averaging over the whole
file (the original rule) let one fat literal drag a small module over
the threshold: a 13-line Express server carrying a 20 KB
const LOGO = 'data:image/png;base64,…' line averages ~1.5 KB/line
and was classified as a bundle, so all ten of its routes were dropped
silently. Dropping the single fattest line barely moves a real
bundle's average (it has several enormous lines, or nothing else at
all) but collapses such a module's to a few dozen bytes.
Blank and comment-only lines are excluded for the same reason in the
other direction: the canonical *.min.js is one enormous line
wrapped in a /*! license */ banner and a //# sourceMappingURL=
trailer. Those decorations are not source, so a bundle that carries
them must still be recognised as a bundle.
A file with no code line left over (a single enormous line, with or without banner/sourcemap decoration) is judged on that line alone — it is the bundle shape by definition.
webpack/rollup output and *.min.js satisfy both conditions; a route
module with a fat inline payload amid real route lines satisfies
neither, so its endpoints survive. Skipping such a file is purely a
parser optimization — small files lex fast regardless — so there is
no need to skip one merely because it embeds a fat literal.
Normalize HTTP method names to standard format
A 22-literal includes? pre-pass used to run ahead of these two
patterns (".get(", ".get (", ... for each verb). Every one of
those literals is . + verb + optional space + (, which
FLEXIBLE_ROUTE_CALL_PATTERN already matches with zero or one
whitespace character — so the pre-pass could never change the
result, and each miss cost 22 Rabin-Karp walks of the whole file.
Replace JS/TS comments with whitespace of the same shape.
Preserves newlines and column offsets so downstream line/column
math (controller_start_line, regex .begin(0), etc.) stays
accurate. Comment bodies are blanked to spaces so a commented-
out decorator like // @Get('/old') never matches the route
regex.
Regex literals get their own state because their bodies routinely
contain quote characters (str.replace(/'/g, "'")). Without it a
single such line inverted the string state for the rest of the file,
which cut both ways: real comments stopped being blanked, so
commented-out routes were reported as live endpoints, and real string
bodies were scanned as code, so a // or /* inside a string opened
a comment that swallowed every route below it.
Every marker below names a directory inside the project
(__tests__/, dist/, vendor/, public/, ...), so they are
matched on the scan-base-relative path. Matched on the absolute
path they also fired on directories ABOVE the base: a checkout
under ~/build/ or ~/vendor/ looked like bundled output and
every route in it disappeared (the JS fixture tree dropped from
430 endpoints to 394, and to 191 under __tests__/).