class

ConstructionKit::AttentionCompoundExec

Inherits ConstructionKit::ExecutableNode < Reference < Object

Attention — stays as compound node for now (RoPE, causal mask, head splitting)

Constructors

new(id : String, d_model : Int32, n_heads : Int32, seq_len : Int32)
Source

Instance methods

adam_mats
Source
backward(output_grads : Hash(String, MicroGPT::Mat)) : Hash(String, MicroGPT::Mat)

Backward pass: receive gradients for outputs, return gradients for inputs

Source
forward(inputs : Hash(String, Tensor)) : Hash(String, Tensor)

Forward pass: read named inputs, produce named outputs

Source
inner
Source
param_count
Source
update(lr : Float64)

Update trainable parameters

Source
weight_mats

Weight access for serialization / checkpoint

Source