class

OpenAI::RealtimeTruncationRetentionRatioTruncation

Inherits JSON::Serializable < Reference < Object

Retain a fraction of the conversation tokens when the conversation exceeds the input token limit. This allows you to amortize truncations across multiple turns, which can help improve cached token usage.

Constructors

new(pull : JSON::PullParser)
Source
new(type_value : RealtimeTruncationRetentionRatioTruncationType = RealtimeTruncationRetentionRatioTruncationType::RetentionRatio, retention_ratio : Float64 = 0.0, token_limits : RealtimeTruncationRetentionRatioTruncationTokenLimits | Nil = nil)
Source
new(*, __pull_for_json_serializable pull : JSON::PullParser)
Source

Instance methods

retention_ratio

Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.

Source
retention_ratio=(retention_ratio : Float64)

Fraction of post-instruction conversation tokens to retain (0.0 - 1.0) when the conversation exceeds the input token limit. Setting this to 0.8 means that messages will be dropped until 80% of the maximum allowed tokens are used. This helps reduce the frequency of truncations and improve cache rates.

Source
token_limits

Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.

Source
token_limits=(token_limits : RealtimeTruncationRetentionRatioTruncationTokenLimits | Nil)

Optional custom token limits for this truncation strategy. If not provided, the model's default token limits will be used.

Source
type_value

Use retention ratio truncation.

Source
type_value=(type_value : RealtimeTruncationRetentionRatioTruncationType)

Use retention ratio truncation.

Source
validate!
Source