class

ML::LLM::Model

Inherits Reference / Object

LLM Model - loads and manages a GGUF model file

Constructors

new(path : String, n_gpu_layers : Int32 = 99)
Source

Instance methods

bos_token
Source
create_context(n_ctx : Int32 = 0, n_batch : Int32 = 512, n_ubatch : Int32 = 0, n_threads : Int32 = 0, flash_attn : Bool = true, embeddings : Bool = false) : Context

Create inference context from this model

Source
description
Source
detokenize(tokens : Array(Int32)) : String

Detokenize tokens to text

Source
eos_token
Source
eot_token
Source
finalize
Source
free
Source
is_eog?(token : Int32) : Bool
Source
n_ctx_train
Source
n_embd
Source
n_heads
Source
n_layers
Source
n_params
Source
path
Source
size_bytes
Source
token_to_piece(token : Int32) : String

Convert token ID to text piece

Source
tokenize(text : String, add_bos : Bool = true) : Array(Int32)

Tokenize text to token IDs

Source
vocab_size

Vocab accessors

Source