class

ML::GGUF::GPUWorkspace

Inherits Reference / Object

Constructors

new(max_seq : Int32, dim : Int32, ffn_dim : Int32, n_heads : Int32, head_dim : Int32, n_experts_used : Int32, rope_cos : Array(Float32), rope_sin : Array(Float32))
Source

Instance methods

attn_out
batched_down_grid
batched_up_grid
cos_cache
dispatch_args
expert_counts
expert_mid
expert_offsets
expert_out
expert_tg_offs
ffn_mid
ffn_out
ffn_out_f32
gate_logits
gather_map
hidden
hidden2
k
moe_input

Pre-allocated MoE batched buffers (contiguous)

moe_mid
moe_output
output
q
qkv
routing_ids
routing_wts
scatter_wts
sin_cache
v
v_t