class

Llama::SamplerChain

Inherits Reference / Object

SamplerChain: manages a chain of samplers, but is not itself a Sampler::Base. Ownership and lifecycle are managed internally.

Constructors

new(no_perf : Bool = false)

Creates a new SamplerChain with optional parameters

Parameters:

  • no_perf: Whether to disable performance counters (default: false)

Raises:

  • Llama::Error if the sampler chain cannot be created
Source

Instance methods

accept(token : Int32)

Accepts a token, updating the internal state of the samplers

Parameters:

  • token: The token to accept
Source
add(sampler : Sampler::Base)

Adds a sampler to the chain

Parameters:

  • sampler: The sampler to add to the chain
Source
finalize

Frees the resources associated with this sampler chain

Source
remove(index : Int)

Removes a sampler from the chain at the given index. Returns the original Sampler::Base instance with ownership restored.

Source
reset

Resets the internal state of the sampler chain. Useful when reusing the same chain for multiple independent generations.

Source
reset_perf

Reset performance counters for this sampler chain

Source
sample(ctx : Context, idx : Int32 = -1) : Int32

Samples a token using the sampler chain

Parameters:

  • ctx: The context to sample from
  • idx: The index of the logits to sample from (-1 for the last token)

Returns:

  • The sampled token
Source
to_unsafe

For C API compatibility

Source