class

Llama::Model

Wrapper for the llama_model structure

Constructors

new(path : String, n_gpu_layers : Int32 = 0, use_mmap : Bool = true, use_mlock : Bool = false, vocab_only : Bool = false)

Creates a new Model instance by loading a model from a file.

Parameters:

path: Path to the model file (.gguf format).
n_gpu_layers: Number of layers to store in VRAM (default: 0). If 0, all layers are loaded to the CPU.
use_mmap: Use mmap if possible (default: true). Reduces memory usage.
use_mlock: Force the system to keep the model in RAM (default: false). May improve performance but increases memory usage.
vocab_only: Only load the vocabulary, no weights (default: false). Useful for inspecting the vocabulary.

Raises:

Llama::Model::Error if the model cannot be loaded.

Source

Instance methods

chat_template(name : String | Nil = nil) : String | Nil

Gets the default chat template for this model

Parameters:

name: Optional template name (nil for default)

Returns:

The chat template string, or nil if not available

Source

classifier?

Returns whether this model is a classifier model

Returns:

true if the model has classifier outputs, false otherwise

Source

cls_label(i : UInt32) : String | Nil

Returns the classifier label by index

Parameters:

i: Index of the classifier output (must be < n_cls_out)

Returns:

The classifier label, or nil if index is out of bounds or no label is provided

Source

context(*args, **options) : Context

Creates a new Context for this model

This method delegates to Context.new, passing self as the model parameter and forwarding all other arguments.

Returns:

A new Context instance

Raises:

Llama::Context::Error if the context cannot be created

Source

decoder_start_token

Returns the token that must be provided to the decoder to start generating output For encoder-decoder models, returns the decoder start token For other models, returns -1

Source

description

Gets a string describing the model type

Returns: