class

Memo::Providers::HTTPPool

Inherits Reference < Object

Posts requests to an embedding API over reused connections, retrying transient failures.

HTTP::Client handles one request at a time, so idle clients are kept in a small pool: concurrent requests each take one (opening a new connection only when none is idle) and return it afterwards. A reused connection skips the TCP and TLS handshakes.

Retries rate limits (429, except an exhausted quota), server errors (500, 502, 503, 504) and network errors, up to max_retries times. Waits as the Retry-After header says, or backs off exponentially (0.5 s, 1 s, 2 s, ... with jitter).

Constants

MAX_IDLE = 8
MAX_RETRY_WAIT = 20.seconds
RETRY_STATUSES = {429, 500, 502, 503, 504}

Constructors

new(base_url : String, max_retries : Int32 = 3)
Source

Instance methods

close

Close idle connections.

Source
max_retries
Source
post(path : String, headers : HTTP::Headers, body : String) : HTTP::Client::Response

POST to path (e.g. "/v1/embeddings"). Returns the final response, which may still be an error, or raises the last network error.

Source