module

Llama

Constants

DEFAULT_SEED = LibLlama::LLAMA_DEFAULT_SEED

==== Native constants (wrapped for user convenience) ====

FILE_MAGIC_GGLA = LibLlama::LLAMA_FILE_MAGIC_GGLA
FILE_MAGIC_GGSN = LibLlama::LLAMA_FILE_MAGIC_GGSN
FILE_MAGIC_GGSQ = LibLlama::LLAMA_FILE_MAGIC_GGSQ
LLAMA_CPP_COMPATIBLE_VERSION = "b#{VERSION}"
LOG_LEVEL_DEBUG = 0

Log level constants (from llama.cpp / ggml)

LOG_LEVEL_ERROR = 3
LOG_LEVEL_INFO = 1
LOG_LEVEL_NONE = 4
LOG_LEVEL_WARNING = 2
SESSION_MAGIC = LibLlama::LLAMA_SESSION_MAGIC
SESSION_VERSION = LibLlama::LLAMA_SESSION_VERSION
TOKEN_NULL = LibLlama::LLAMA_TOKEN_NULL
VERSION = {{ (`shards version /tmp/tmp.nMHmkj/src/src`).chomp.stringify }}

Class methods

apply_chat_template(template : String | Nil, messages : Array(ChatMessage), add_assistant : Bool = true) : String

Applies a chat template to a list of messages

Parameters:

  • template: The template string (nil to use model's default)
  • messages: Array of chat messages
  • add_assistant: Whether to end with an assistant message prefix

Returns:

  • The formatted prompt string

Raises:

  • Llama::Error if template application fails
Source
builtin_chat_templates

Gets the list of built-in chat templates

Returns:

  • Array of template names
Source
error_message(code : Int32) : String
Source
format_error(message : String, code : Int32 | Nil = nil, context : String | Nil = nil) : String
Source
generate(model_path : String, prompt : String, max_tokens : Int32 = 128, temperature : Float32 = 0.8) : String

Generates text from a prompt using a model

This is a convenience method that loads a model, creates a context, and generates text in a single call.

response = Llama.generate(
  "/path/to/model.gguf",
  "Once upon a time",
  max_tokens: 100,
  temperature: 0.7
)
puts response

Parameters:

  • model_path: Path to the model file (.gguf format)
  • prompt: The input prompt
  • max_tokens: Maximum number of tokens to generate (must be positive)
  • temperature: Sampling temperature (0.0 = greedy, 1.0 = more random)

Returns:

  • The generated text

Raises:

  • ArgumentError if parameters are invalid
  • Llama::Model::Error if model loading fails
  • Llama::Context::Error if text generation fails
Source
init

Thread-safe, idempotent initialization of the llama.cpp backend. You do not need to call this manually in most cases.

Source
log_level

Get the current log level

Returns:

  • The current log level
Source
log_level=(level : Int32)

Set the log level

Parameters:

  • level : Int32 - log level (0=DEBUG, 1=INFO, 2=WARNING, 3=ERROR, 4=NONE)

Example: Llama.log_level = Llama::LOG_LEVEL_ERROR # Only show errors Llama.log_level = Llama::LOG_LEVEL_NONE # Disable all logging

Source
log_set

Set a custom log callback

The block receives:

  • level : Int32 - log level (0=DEBUG, 1=INFO, 2=WARNING, 3=ERROR)
  • message : String - log message

Example: Llama.log_set do |level, message| if level >= Llama::LOG_LEVEL_ERROR STDERR.print message end end

Source
max_parallel_sequences

Returns the maximum number of parallel sequences supported by backend This is a thin wrapper around LibLlama.llama_max_parallel_sequences.

Source
measure_ms

Measures elapsed time in milliseconds for a block using llama.cpp's clock.

elapsed = Llama.measure_ms do
  # ... code to measure ...
end
puts "Elapsed: #{elapsed} ms"

Returns:

  • Float64: elapsed milliseconds
Source
process_escapes(text : String) : String

Process escape sequences in a string

This method processes common escape sequences like \n, \t, etc. in a string, converting them to their actual character representations.

text = Llama.process_escapes("Hello\\nWorld")
puts text # Prints "Hello" and "World" on separate lines

Parameters:

  • text: The input string containing escape sequences

Returns:

  • A new string with escape sequences processed
Source
system_info

Returns the llama.cpp system information

This method provides information about the llama.cpp build, including BLAS configuration, CPU features, and GPU support.

info = Llama.system_info
puts info

Returns:

  • A string containing system information
Source
time_ms

Returns the current time in milliseconds since the Unix epoch (llama.cpp compatible).

t0 = Llama.time_ms
# ... some processing ...
t1 = Llama.time_ms
elapsed = t1 - t0
puts "Elapsed: #{elapsed} ms"

Returns:

  • Int64: milliseconds since epoch
Source
time_us

Returns the current time in microseconds since the Unix epoch (llama.cpp compatible).

This is a high-level wrapper for LibLlama.llama_time_us.

t0 = Llama.time_us
# ... some processing ...
t1 = Llama.time_us
elapsed_ms = (t1 - t0) / 1000.0
puts "Elapsed: #{elapsed_ms} ms"

Returns:

  • Int64: microseconds since epoch
Source
tokenize_and_format(vocab : Vocab, text : String, add_bos : Bool = true, parse_special : Bool = true, ids_only : Bool = false) : String

Tokenize text and return formatted output

This is a convenience method that tokenizes text and returns a formatted string representation of the tokens.

model = Llama::Model.new("/path/to/model.gguf")
result = Llama.tokenize_and_format(model.vocab, "Hello, world!", ids_only: true)
puts result # Prints "[1, 2, 3, ...]"

Parameters:

  • vocab: The vocabulary to use for tokenization
  • text: The text to tokenize
  • add_bos: Whether to add BOS token (default: true)
  • parse_special: Whether to parse special tokens (default: true)
  • ids_only: Whether to return only token IDs (default: false)

Returns:

  • A formatted string representation of the tokens
Source
uninit

Thread-safe, idempotent finalization of the llama.cpp backend. Call this if you want to explicitly release all backend resources before program exit.

Source

Nested types