class

Memo::QueryCache

Inherits Reference < Object

LRU query embedding cache with optional DB persistence.

Avoids repeated API calls for the same query text by caching the embedding vector. Memory LRU is checked first (instant), then DB (fast), then API (slow).

Memory is bounded by max_entries (LRU eviction). DB is bounded by max_db_entries (oldest evicted on prune).

Constructors

new(max_entries : Int32 = 10000, max_db_entries : Int32 = 100000, db : DB::Database | Nil = nil, service_id : Int64 = 0)
Source

Instance methods

clear

Clear all cached entries (memory and DB)

Source
get(query : String) : Tuple(Array(Float64), Int32) | Nil

Look up a cached embedding for a query string. Returns {embedding, token_count} or nil on miss.

Source
hit_rate

Cache hit rate as a percentage

Source
hits
Source
max_db_entries
Source
max_entries
Source
misses
Source
put(query : String, embedding : Array(Float64), token_count : Int32)

Store an embedding for a query string.

Source
size

Number of entries in memory cache

Source