module

Smith::Context

Keeps the conversation transcript from growing past the provider's context window.

The invariant everything here is built around: every tool_use block must keep exactly one matching tool_result, and vice versa. Anthropic and OpenAI reject a request outright if that pairing is broken, so compaction may shorten content but must never orphan half a pair.

Constants

TRUNCATED_RESULT_BYTES = 2000

What a tool result gets shortened to in stage 1.

Class methods

compact(messages : Array(LLM::Message), max_tokens : Int32, &summarize : Array(LLM::Message) -> String) : Result

Brings messages under max_tokens if it can, in two escalating stages.

summarize receives the prefix being dropped and returns a replacement text. It is a callback rather than a provider so this whole module stays testable without a network.

Source
compact(messages : Array(LLM::Message), max_tokens : Int32) : Result

Convenience wrapper for callers that have no summarizer available.

Source
estimate_text_tokens(text : String) : Int32

The same heuristic applied to a plain string, so a breakdown of the context and the compaction decision cannot drift apart.

Source
estimate_tokens(messages : Array(LLM::Message)) : Int32

Rough heuristic, not an exact count. The point is to react before the limit, not to spend the last token of it.

Source
safe_cut_index(messages : Array(LLM::Message), max_tokens : Int32) : Int32

The lowest index of a User message such that everything from there on fits the budget. User messages only ever start a turn (Agent#send), so cutting there can never orphan a tool pair. Returns 0 when no such cut exists — the caller then leaves the history alone rather than breaking it.

Source

Nested types