class

ML::NN::LayerNorm

Layer Normalization: y = (x - mean) / sqrt(var + eps) * gamma + beta

Constructors

new(dim : Int32, eps : Float32 = 1e-5_f32, device : Tensor::Device = Tensor.default_device)

Convenience constructor for single dimension

new(normalized_shape : Array(Int32) | Int32, eps : Float32 = 1e-5_f32, device : Tensor::Device = Tensor.default_device)

bias

call(x : Autograd::Variable) : Autograd::Variable

eps

forward(x : Autograd::Variable) : Autograd::Variable

Forward pass x: [..., *normalized_shape] output: same shape as x

normalized_shape

parameters

Get all trainable parameters

weight