module

ML::GGUF::ComputeBackend

Abstract compute backend. All math ops go through this interface.

Instance methods

dot(a : Array(Float32), a_off : Int32, b : Array(Float32), b_off : Int32, len : Int32) : Float32

Dot product for attention scores

Source
gelu(x : Float32) : Float32

GELU activation (returns new value)

Source
layer_norm!(x : Array(Float32), n_pos : Int32, dim : Int32, w : Array(Float32), b : Array(Float32)) : Nil

In-place layer norm

Source
matmul(x : Array(Float32), rows : Int32, qw : QuantWeight, bias : Array(Float32)) : Array(Float32)

Matrix multiply: x[rows, in_dim] × W_quant + bias → [rows, out_dim]

Source
softmax_row!(scores : Array(Float32), offset : Int32, len : Int32) : Nil

In-place softmax over a row

Source