LLM::General
General OpenAI-compatible LLM client
Constants
How OpenAI, vLLM, Anthropic, Gemini and LM Studio state the window ...
... and the prompt's real size, which corrects noir's chars/4 estimate.
Reasoning models answer a non-default temperature with HTTP 400:
OpenAI's o-series and GPT-5 family (its -chat models excepted), and
Claude from Opus 4.7 / Sonnet 5 / Haiku 5.5 on (Fable and Mythos
included). post recovers from that rejection for any model, so this
list only saves the first wasted request for the models it names.
A bad key, a missing model, a wrong URL or an unreachable host fails every request the same way. After this many in a row, before the provider has answered once, the client stops sending, so a 1000-file scan does not repeat the same failure 1000 times; the files it skips are reported as unanalyzed by the caller. Once it has answered, a failure is an outage or a slow prompt (a read timeout looks the same as a refused connection here) and every request keeps its own chance.
--ai-provider names that stand for a fixed endpoint. CLI validation
rejects any other bare name, so a typo fails before the scan.
Constructors
Class methods
The request URL for a provider given as a URL; LLM::Anthropic
resolves it to the Messages endpoint instead.
Decided on the URI path, not the whole string: an Azure-style
...?api-version=2024-02-01 query used to get /chat/completions
appended after it.
A streamed reply folded back into the non-streamed shape the callers
read. A body that is not SSE (a server that ignored stream, or an
HTML page) is parsed as-is so it fails loudly, and an error event is
returned as the whole reply. A stream that ends without a finish
reason was cut off, and says so.
nil when the rejection is not a context overflow. A 413 is one even with no wording: a reverse proxy's body-size cap answers in HTML.
Parse provider response into normalized JSON action payload. If the model returns tool_calls, convert them to: {"action":"<function_name>","args":{...}} Otherwise, return cleaned textual content as-is.
A server that implements only json_object, or no structured output
at all, answers every json_schema request with a 400 naming it.
--ai-stream: endpoint-extraction requests ask for SSE, so a proxy
that drops idle connections sees bytes while the model generates.
--ai-stream: endpoint-extraction requests ask for SSE, so a proxy
that drops idle connections sees bytes while the model generates.
The 400 for a rejected sampling parameter (temperature, or the
seed from --ai-seed): OpenAI names it in param
(unsupported_value), Anthropic in the message. Only the error itself
is read; a gateway that echoes the request would otherwise match on
every 400.