class

ML::Optim::Adam

Inherits Reference / Object

Adam optimizer with optional weight decay (AdamW)

Constructors

new(params : Array(Autograd::Variable), lr : Float32 = 0.001_f32, beta1 : Float32 = 0.9_f32, beta2 : Float32 = 0.999_f32, eps : Float32 = 1e-8_f32, weight_decay : Float32 = 0.0_f32, amsgrad : Bool = false)
Source
new(param_groups : Array(ParamGroup), **kwargs) : Adam

Create with parameter groups

Source

Instance methods

add_param_group(group : ParamGroup) : Nil

Add parameter group

Source
amsgrad
Source
amsgrad=(amsgrad : Bool)
Source
beta1
Source
beta1=(beta1 : Float32)
Source
beta2
Source
beta2=(beta2 : Float32)
Source
eps=(eps : Float32)
Source
load_state_dict(dict : Hash(String, Tuple(Tensor, Tensor, Int32))) : Nil

Load state dict

Source
lr=(lr : Float32)
Source
parameters

Get all parameters

Source
state_dict

State dict for checkpointing

Source
step

Single optimization step

Source
weight_decay
Source
weight_decay=(weight_decay : Float32)
Source
zero_grad

Zero all gradients

Source