module

ML::GGUF::Dequant

Constants

K_SCALE_SIZE = 12
QK_K = 256

Class methods

dequantize(data : Bytes, type : TensorType, n_elements : Int32) : Array(Float32)

Main entry: dequantize raw bytes to Float32 array

Source
dequantize_f16(data : Bytes, n : Int32) : Array(Float32)

F16: convert half-float to float32

Source
dequantize_f32(data : Bytes, n : Int32) : Array(Float32)

F32: direct copy

Source
dequantize_q5_k(data : Bytes, n : Int32) : Array(Float32)

Q5_K: 5.5 bits per weight, QK_K=256 block, 176 bytes/block Block layout: [d:f16][dmin:f16][scales:12B][qh:32B][qs:128B]

Source
dequantize_q6_k(data : Bytes, n : Int32) : Array(Float32)

Q6_K: 6.5625 bits per weight, QK_K=256 block, 210 bytes/block Block layout: [ql:128B][qh:64B][scales:16B][d:f16]

Source
fp16_to_f32(bytes : Bytes) : Float32

Convert IEEE 754 half-precision (2 bytes LE) to Float32

Source
fp16_to_f32(h : UInt16) : Float32
Source
get_scale_min_k4(j : Int32, scales : Pointer(UInt8)) : Tuple(Float32, Float32)

Extract 6-bit scale and min from packed scales array (K_SCALE_SIZE=12 bytes) Ported from get_scale_min_k4() in ggml-quants.c

Source