class

Llama::Sampler::TopNSigma

Inherits Llama::Sampler::Base / Reference / Object

Top-N Sigma sampler

The Top-N Sigma sampler selects tokens based on their distance from the mean in terms of standard deviations. This is based on the paper "Top-nσ: Not All Logits Are You Need" (https://arxiv.org/pdf/2411.07641).

Example:

sampler = Llama::Sampler::TopNSigma.new(2.0) # Keep tokens within 2 standard deviations

Constructors

new(n : Float32)

Creates a new Top-N Sigma sampler

Parameters:

  • n: Number of standard deviations to keep

Raises:

  • Llama::Error if the sampler cannot be created
Source