class

ML::NN::RMSNorm

Inherits Reference / Object

RMSNorm - simplified LayerNorm without mean centering Used in some modern architectures (LLaMA, etc.)

Constructors

new(dim : Int32, eps : Float32 = 1e-5_f32, device : Tensor::Device = Tensor.default_device)
Source

Instance methods

call(x : Autograd::Variable) : Autograd::Variable
Source
forward(x : Autograd::Variable) : Autograd::Variable
Source
parameters
Source
weight
Source