class

Llama::Sampler::Typical

Inherits Llama::Sampler::Base / Reference / Object

Typical sampler

The Typical sampler selects tokens based on their "typicality" (entropy). It filters out tokens that are either too predictable or too surprising, leading to more natural and diverse text generation.

Based on the paper: https://arxiv.org/abs/2202.00666

Example:

sampler = Llama::Sampler::Typical.new(0.95, 1) # Keep tokens with typicality >= 0.95

Constructors

new(p : Float32, min_keep : Int32 = 1)

Creates a new Typical sampler

Parameters:

  • p: The typicality threshold (0.0 to 1.0)
  • min_keep: Minimum number of tokens to keep

Raises:

  • Llama::Error if the sampler cannot be created
Source