Noir::JSRouteExtractor
JSRouteExtractor provides a unified interface for extracting routes from JavaScript files
Constants
Pre-filter for extract_routes: returns false when content
contains no shape the JS parser knows how to emit (any verb
invocation pattern like .get(/.post(/... or Fastify/Restify
.route(, plus Express-style mounts .use( which feed into the
cross-file router prefix table). Matching is millions of times
cheaper than tokenizing the file.
Client-side UI framework imports. A file that imports a browser
UI framework (Vue, React, Angular, Svelte, Solid, Preact) and its
satellite libs (pinia, vue-router, @vueuse, react-router, ...) is
SPA/frontend code, not an HTTP server. Its route-shaped calls are
outbound API-client requests against a configured client — e.g.
directus's admin app does api.get(/users/${userId}) where api
is a wrapped axios instance imported from @/api. The existing
axios/got/ky markers miss these because the wrapper hides the raw
client behind a local module, but the UI-framework import is an
unambiguous "this is browser code" signal. directus's admin SPA
alone parks ~61 phantom Express endpoints across
app/src/{stores,composables,layouts,...} this way. Like the
test-stub markers, this is gated by the HTTP-server-import
exemption below: an SSR entrypoint that imports BOTH vue and
express keeps its routes.
Bytes after a route's start searched for its declaration when the
direct .verb( probe misses. The declaration is the call at the
route's own position; searching the whole file found the first route
elsewhere with the same path (wrong handler) and cost O(file) per route.
A @fastify/autoload plugin module names no framework at all — it
receives the instance as a parameter:
export default async function (fastify) {
fastify.get('/status', handler)
}
so no import marker fires and the shared extractor happily reads those
registrations for whichever framework asked. Express reported /status
and /go out of the fastify autoload fixture, without the autoPrefix
the Fastify analyzer applies. The receiver name is the only evidence in
the file, and fastify is unambiguous — nothing else calls its app
instance that. Consulted only when the file imports no HTTP server.
Mirrors ExpressAuthTagger::REFUSING_DEFINITION: a definition whose only
response is a 401/403 refusal (or that calls bare next() without
sending a non-refusal response) is a middleware/guard, not the route's
handler, so its body must not pollute the route's params or callees.
Constructors that build an HTTP client. client.get('/todo', cb) is
the same call shape as a route registration, so a client module reads as
a server to the shared extractor: the Express analyzer reported
GET /todo and DELETE /todo/example out of a restify-clients module
that only calls a remote API.
The restify analyzer already refused these; the check belongs here so
every framework sharing the extractor gets it. It is deliberately
limited to client constructors rather than client package names —
a genuine route file may well require('axios') to call downstream
services, and gating on that would drop its routes.
Real HTTP-server library imports. When any of these is present alongside a test-stub marker, the file is doing legitimate server work (e.g., spinning up a test instance of an Express app) and we still want to extract its routes.
The hyphen forms are also ordinary English in production router names
(routes/ab-test.js, routes/api-spec.js), so they count as a test
only without an HTTP server import, or with a test harness (a supertest
suite imports express too). NestJS's app.e2e-spec.ts is always a test.
A function literal: function, x =>, or a parameter list then =>
(async (ctx: Context<{ id: string }>) => …). (a || b) is not one.
Ceiling on the prefixes one router collects. The (router, prefix) dedupe already bounds the work; this only stops a diamond lattice of parallel edges from multiplying without end. A shared router mounted under many paths stays well below it. Shared with JSParser's same-file router resolution.
Average bytes-per-line above which a file is considered dominated by long lines, i.e. a bundle rather than hand-written source that merely carries one fat literal (a big inline JSON seed, an embedded base64 data URI, a long regex). Real code keeps the average low because it has many short lines around any such literal.
Byte length above which a single source line is considered "long".
Hand-written JS/TS keeps lines well under this even in dense route
tables (noir's own widest fixture line is ~150 bytes); webpack/
rollup/esbuild bundles and *.min.js assets routinely pack tens of
thousands of bytes onto one line, so 5000 leaves a wide margin.
NB: the metric is bytes, not characters — a dense single-line
non-Latin blob (>=5000 bytes but fewer chars) can trip it, which is
acceptable since real route registrations are ASCII verbs/paths.
multer (and @koa/multer) middleware names the multipart file fields:
upload.single('avatar'), upload.array('photos', 3),
upload.fields([{ name: 'icon' }]).
formidable hands the parsed multipart body to a callback, form.parse(req, (err, fields, files) => …), or resolves it, const [fields, files] = await form.parse(req): files.image is an uploaded file, fields.title
a form field.
import { A, B as C } from './x', const { A, B: C } = require('./x').
Sibling JS/TS server frameworks that DON'T call extract_routes
(NestJS uses decorators, Hapi/Elysia/AdonisJS have their own
tree-sitter extractors) but whose files are still walked by every
JS/TS analyzer's parallel_file_scan. These markers are exclusion-
only: recognizing "this file belongs to NestJS" keeps Express/
Fastify/Koa/Hono/Restify from re-extracting a route-shaped call
(an inline example, a raw app.use() bridge, ...) out of it.
True when content carries definitive import evidence of a
different shared-extractor (or sibling) framework than
framework, with no evidence of framework itself. Guards the main
extract_routes call in Express/Fastify/Koa/Hono/Restify's
analyzers: a file that only imports 'hono' should never be
re-attributed to js_express just because its .get()/.post()
chaining looks the same as Express's (issue #2368) — whichever
analyzer runs over it first no longer matters once #2367 made the
dedup tiebreak deterministic, because the over-matching analyzer
never produces a competing endpoint in the first place.
Files with no shared-extractor/sibling import at all (a router
module that only receives app/router as a bare parameter, with
no import in that file) return false here and fall through to the
existing, permissive whole-tree scan — there's no import to
disambiguate on, so narrowing further is left as follow-up scope
(a confirmed package.json dependency or cross-file mount signal
would be needed to resolve those).
Per-framework precompiled unions of the two tables above: the
framework's own import markers, and the markers of every other
shared-extractor framework. Together with the sibling union this
turns the up-to-44-literal includes? walk below into three
matches. Five analyzers call this on every JS/TS file, so the old
shape scanned some trees over 200 times per file.
): void {, ): Response {, ): string[] {: a return type with no
brackets of its own, so the next { opens the body.
delete req.query.lang drops an input and req.body.sort = 'asc' or
req.headers['x-tenant'] = 'default' sets one; neither reads what the
client sent. req.query.page ||= '1' does read it, and is kept.
A router factory (module.exports = () => { router.get('/:id', …); return router }) sends inside the routes it registers; it is a
mount, not a handler.
Any import of a server framework the shared extractor serves. A file
without one reads this.post(...) as an outbound client call, not a
route (JSParser#initialize).
Import markers for the five frameworks whose analyzers call
extract_routes directly and therefore share its framework-agnostic
verb-chaining shape (.get(/.post(/...). Keyed by the same Symbol
each analyzer passes to other_shared_extractor_framework? below.
Extract static path declarations from JavaScript content
Returns array of hashes with static_path (URL prefix) and file_path (directory)
framework scopes the scan to one framework's static-mount idiom so a
framework analyzer running over a sibling project's file (every JS
analyzer walks all .js/.ts files) doesn't pick up another
framework's static declaration and re-emit it under the wrong tech.
nil runs every pattern (back-compat for un-scoped callers).
True when the file's route-shaped calls are almost certainly mock-server stubs (Ember pretender, MSW, nock, ...) rather than real route registrations. Two routes:
Path markers strict enough that the HTTP-server-import
exemption shouldn't apply: /e2e/, /cypress/, /playwright/,
/__mocks__/, /__tests__/, /e2e-tests/, /mirage/. Real
apps never park production handlers under any of these — even
when the harness file imports express to spin up a faked
service (Ghost's e2e/helpers/services/stripe/fake-stripe-server.ts
is the canonical example). Keeping the exemption out of these
paths catches the harness fakes without affecting legit
backend code.
const NAME = 'lit', and process.env.X || 'lit' (?? too): the
literal is what the app serves when the variable is unset.
The literal must end the expression: '/api/' + version is no constant.
A suite file that registers its own throwaway app is a test whatever
server library it imports. Line-anchored: .describe( is also zod's
and Joi's schema annotation in ordinary route files.
- Filename markers fire unconditionally —
foo.test.tsis a test no matter what it imports. - Strict path markers also fire unconditionally —
e2e/,cypress/, etc. are dedicated test/mock trees that never contain production handlers, even when the harness file imports a server lib. - Library + the remaining directory markers honor an
exemption — if the file also imports a real HTTP server
lib (express, fastify, ...), keep it so legit test-server
harnesses (e.g. mattermost's
webhook_serve.js) keep their routes.include_client_frameworkscontrols whether a client-side UI framework import (Vue/React/...) counts as a skip signal. It must be ON for the verb-DSL extractor (a React/Vue file callingapi.get(...)is an outbound client call, not a route), but OFF for analyzers whose OWN route definitions live in client-side files — TanStack Router (createFileRoute) and tRPC route modules routinelyimport { ... } from 'react', and skipping them on that basis dropped every such route. The test-stub library markers (msw/supertest/...) and path/ filename markers still apply in both modes. Precompiled unions of the marker lists above.Regex.unionescapes every String argument, so each is exactly theany? includes?it replaces — but the content lists are long (72 test-stub libraries, 32 client frameworks, 26 server libraries), and every JS/TS file in the tree used to be walked once per literal.
Hard test-file markers: when the filename itself follows a
ubiquitous test convention, the file practically never
defines real routes. Skip these even when the file imports
a real HTTP server lib — NestJS e2e tests routinely import
@nestjs/platform-express for type-only references, and
supertest harnesses import the same modules they exercise.
The supertest request(app).get(...) shape would otherwise
ride the HTTP-server-import exemption straight back into the
parser.
Test-fixture libraries whose API mimics route registration:
pretender/miragejs expose server.get("/x", ...), MSW and
nock expose handler builders, sinon-via-faker likewise. When
these libraries are imported, virtually every route-shaped call
in the file is a stub, not a real registration. Substring match
is enough — these tokens never appear in production HTTP server
source under normal circumstances.
Path-level evidence that a file is a mock-server fixture.
Pretender helpers in particular get a helper/this arg and
call this.get(...) / this.post(...) directly, so they have
no library-name imports the content filter can hook on — fall
back to the convention-based filename match.
A Next.js pages-router tRPC mount: a [trpc] / [...trpc] file (the
adapter reads the procedure path from that query key) handing a router
to createNextApiHandler. Its URLs are the router's procedures, which
ts_trpc emits, not a {trpc} catch-all.
Deno/mocha-style foo_test.ts / foo_spec.js. snake_case is ordinary
in production names too (routes/ab_test.js, convex/load_test.ts,
speed_test.js), so these are a test only when the file holds a suite
(TEST_SUITE_MARKER) — never on the name alone and never because of an
import (a fastify-plugin or router module imports nothing a test would).
Class methods
Callees of the route-specific steps before the handler. In Express's
handler-chain style they do part of the route's work: juice-shop's
app.post('/file-upload', …, checkFileType, handleZipFileUpload, handleXmlUpload, handleYamlUpload) extracts the zip and parses the
XML in steps the last argument never reaches. Only steps that resolve
to app source count (uploadToMemory.single('file') and
express.json() do not), a guard step reads as no callees
(read_handler_facts), and the caller adds these after the handler's
own. Steps nearest the handler come first, so under
Callee::MAX_PER_ENDPOINT they outrank the leading validators.
The body of the function defined at line of content, its signature
allowed to wrap lines (function v(\n ctx,\n): Promise<{ a: A }> {):
the first { after the last => on the line, or after the parameter
list, that follows ) or > — braces inside a return type follow <
or :. A concise body (= () => jwt({ … })) yields the definition line.
The parameter names of the function defined at line (function f(req, res), f = async (req, res) =>, f = req =>); "" for a destructured
or rest parameter.
Local binding => the names it is exported under. An importer that
takes a router by name (const { usersRouter } = require('./users'),
import { usersRouter } from './users', users.router) records the
mount prefix under that exported name, and the routes are registered
on the local binding, so the two have to be joined here. Covers
export const x = …, export { a as b }, module.exports = { a, b: c },
export default { a, b: c }
and (module.)exports.b = a.
Every offset here is a BYTE offset into offsets.content (see
ByteOffsets) so a route's cost is local to its own text.
The file's own string constants plus those it imports by name from a
relative module (import { BASE_URL } from './config'); local wins.
finale-rest (and its predecessor epilogue) finale.resource({ endpoints: ['/users', '/users/:id'] }) registers list + create on the first path
and read + update + delete on the second (the first again when there
is no second). A ${name} placeholder is expanded when an enclosing
for (const { name } of models) walks an array literal declared in the
same file (juice-shop's autoModels); any other placeholder drops the
path. Returns {method, url, call offset}; pass comment-stripped source.
Delegate to JSLiteralScanner for literal-aware brace matching
Delegate to JSLiteralScanner for literal-aware paren matching
The route argument that serves the request, as {source, line it starts
on, index, whether it precedes the renderer}: the last one, unless the last one is a shared renderer
(router.get('/', load, respond): respond sends whatever the step
before it left on res.locals, and ends many registrations). Then it
is the step before it, when that step is the route's own work: an
inline closure, or a definition that writes res.locals, that calls
next() and is not a guard. A pass-through like noCache is not.
Facts for an inline closure passed directly to app.use('/path', (req, res) => { … }).
Returns nil when source is not a function expression or arrow closure.
The closure written inline as the handler step (router.get('/x', auth(), async (ctx) => {…}), or inside a transparent wrapper:
asyncHandler(async (req, res) => {…})) has no name to resolve; it is
the handler, at the line it starts on.
1-based line number for a CHAR index into content.
True when content looks like a minified/bundled asset rather than
hand-written source. Two conditions must BOTH hold so we never drop
the routes of a normal file that just happens to carry one long
line (issue #1903 review):
- at least one line reaches MINIFIED_LINE_THRESHOLD bytes, and
- the rest of the file — every code line except that single fattest one — still averages MINIFIED_AVG_LINE_THRESHOLD bytes or more, i.e. long lines dominate and newline density is low.
Condition (2) deliberately sets aside the longest line and every
blank/comment-only line before averaging. Averaging over the whole
file (the original rule) let one fat literal drag a small module over
the threshold: a 13-line Express server carrying a 20 KB
const LOGO = 'data:image/png;base64,…' line averages ~1.5 KB/line
and was classified as a bundle, so all ten of its routes were dropped
silently. Dropping the single fattest line barely moves a real
bundle's average (it has several enormous lines, or nothing else at
all) but collapses such a module's to a few dozen bytes.
Blank and comment-only lines are excluded for the same reason in the
other direction: the canonical *.min.js is one enormous line
wrapped in a /*! license */ banner and a //# sourceMappingURL=
trailer. Those decorations are not source, so a bundle that carries
them must still be recognised as a bundle.
A file with no code line left over (a single enormous line, with or without banner/sourcemap decoration) is judged on that line alone — it is the bundle shape by definition.
webpack/rollup output and *.min.js satisfy both conditions; a route
module with a fat inline payload amid real route lines satisfies
neither, so its endpoints survive. Skipping such a file is purely a
parser optimization — small files lex fast regardless — so there is
no need to skip one merely because it embeds a fat literal.
The handler step (handler_step) resolved to a definition when it is a
plain reference (login, users.create, auth()), possibly in a
transparent wrapper (asyncHandler(login)); nil for an inline closure.
unwrap_guards: the caller's auth tagger reports a refusing wrapper
(withAuth(list)) as the guard, so the handler is what it wraps.
Normalize HTTP method names to standard format
Pushes mount prefixes along {parent, prefix, child} edges into
prefixes, starting from each parent's existing prefixes (or
fallback when it has none, without recording that on the parent).
Depth-first with the routers on the current mount path excluded, so a
self-mount (router.use('/v1', router.routes())), a plugin that
registers itself, or an a -> b -> a cycle is expanded at most once
instead of growing the prefix list without bound — the fixpoint loops
this replaced appended to the array they were iterating.
Returns true when some router hit MAX_MOUNT_PREFIXES and lost prefixes.
A template-literal mount path (`/${rootPath}`) as a rooted path.
Substitutes every ${NAME} that constants knows; the rest stay.
A 22-literal includes? pre-pass used to run ahead of these two
patterns (".get(", ".get (", ... for each verb). Every one of
those literals is . + verb + optional space + (, which
FLEXIBLE_ROUTE_CALL_PATTERN already matches with zero or one
whitespace character — so the pre-pass could never change the
result, and each miss cost 22 Rabin-Karp walks of the whole file.
Whether a resolved step serves the request: not a guard, and not a
pass-through that only calls next() (router.all('/api/*', prepareAPI) registers no handler). Unknown facts keep the step.
The step before a shared renderer (handler_step) passes on by design.
Replace JS/TS comments with whitespace of the same shape.
Preserves newlines and column offsets so downstream line/column
math (controller_start_line, regex .begin(0), etc.) stays
accurate. Comment bodies are blanked to spaces so a commented-
out decorator like // @Get('/old') never matches the route
regex.
Regex literals get their own state because their bodies routinely
contain quote characters (str.replace(/'/g, "'")). Without it a
single such line inverted the string state for the rest of the file,
which cut both ways: real comments stopped being blanked, so
commented-out routes were reported as live endpoints, and real string
bodies were scanned as code, so a // or /* inside a string opened
a comment that swallowed every route below it.
Whether the file is a test by its name, a dedicated test tree, or a
suite under test//tests/, ignoring what it imports. Callers whose
frameworks HTTP_SERVER_LIBRARY_MARKER does not list (hapi, Elysia)
use this instead of test_stub_only?, which would let any server
import keep a test's routes and drop real routes beside axios/react.
The content-free half of test_file?: a test by its name
(*.test.*, *.spec.*, ...) or inside a dedicated test tree
(__tests__/, e2e/, ...). Shared with JavascriptEngine#js_test_file?,
which has to decide before it reads the file.
Every marker below names a directory inside the project
(__tests__/, dist/, vendor/, public/, ...), so they are
matched on the scan-base-relative path. Matched on the absolute
path they also fired on directories ABOVE the base: a checkout
under ~/build/ or ~/vendor/ looked like bundled output and
every route in it disappeared (the JS fixture tree dropped from
430 endpoints to 394, and to 191 under __tests__/).