struct

Memo::Config::Chunking

Inherits Struct < Value < Object

Chunking configuration for text segmentation

The app's preferred chunk sizes (must be <= service max_tokens)

Constructors

new(min_tokens : Int32 = 200, max_tokens : Int32 = 2000, no_chunk_threshold : Int32 = 1500, tokens_per_byte : Float64 = 0.25)
Source

Instance methods

max_tokens
Source
min_tokens
Source
no_chunk_threshold
Source
tokens_per_byte
Source
with_tokens_per_byte(ratio : Float64) : Chunking

Return a copy with updated tokens_per_byte ratio

Source