Num::NN
Instance methods
Computes a 2D convolution over input images. Intended to be used in 2d convolution forward pass. This applies a 2D cross-correlation, not to be confused with the mathematical convolution.
Arguments
- input :
Tensor- 4DTensorbatch of images of the size [N,C_in,H_in,W_in] - weight :
Tensor- 4DTensorconvolving kernel weights of the size [C_out,C_in,kH,kW] - bias :
Tensor- 3DTensorbias of the size [C_out,1,1] - padding :
Tuple-Tuplewith height and width of the padding - stride :
Tuple-Tuplewith height and width of the stride
Computes gradients of a 2D convolution. Intended to be used after
conv2d to calculate gradients in backward pass.
Arguments
- input :
Tensor- 4DTensorbatch of images of the size [N,C_in,H_in,W_in] - weight :
Tensor- 4DTensorconvolving kernel weights of the size [C_out,C_in,kH,kW] - bias :
Tensor- 3DTensorbias of the size [C_out,1,1] - grad_output :
Tensor- 4DTensorgradient of size [N, C_out, H_out, W_out] - padding :
Tuple-Tuplewith height and width of the padding - stride :
Tuple-Tuplewith height and width of the stride
Computes a forward dropout activation
Arguments
- input :
Tensor-Tensorto activate - mask :
Tensor- Mask to dropout - probability :
Float- Probability of dropout
Computes a forward dropout activation
Arguments
- input :
Tensor-Tensorto activate - mask :
Tensor- Mask to dropout - probability :
Float- Probability of dropout
Computes a backwards dropout derivative
Arguments
- gradient :
Tensor-Tensorused to compute backwards pass - mask :
Tensor- Mask to apply to the gradient - probability :
Float- Probability of dropout
Exponential linear unit activation
Arguments
- x :
Tensor-Tensorto activate
Exponential linear unit activation
Arguments
- x :
Tensor-Tensorto activate
ELU derivative
Arguments
- gradient :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Computes a 2D convolution over input images. Intended to be used in 2d convolution forward pass. This applies a 2D cross-correlation, not to be confused with the mathematical convolution.
Arguments
- input :
Tensor- 4DTensorbatch of images of the size [N,C_in,H_in,W_in] - weight :
Tensor- 4DTensorconvolving kernel weights of the size [C_out,C_in,kH,kW] - bias :
Tensor- 3DTensorbias of the size [C_out,1,1] - padding :
Tuple-Tuplewith height and width of the padding - stride :
Tuple-Tuplewith height and width of the stride
Computes gradients of a 2D convolution. Intended to be used after
conv2d to calculate gradients in backward pass.
Arguments
- input :
Tensor- 4DTensorbatch of images of the size [N,C_in,H_in,W_in] - weight :
Tensor- 4DTensorconvolving kernel weights of the size [C_out,C_in,kH,kW] - bias :
Tensor- 3DTensorbias of the size [C_out,1,1] - grad_output :
Tensor- 4DTensorgradient of size [N, C_out, H_out, W_out] - padding :
Tuple-Tuplewith height and width of the padding - stride :
Tuple-Tuplewith height and width of the stride
Leaky ReLU activation function
Arguments
- x :
Tensor- Argument to activate
Leaky ReLU activation function
Arguments
- x :
Tensor- Argument to activate
Leaky ReLU activation function
Arguments
- x :
Tensor- Argument to activate
Leaky ReLU activation function
Arguments
- x :
Tensor- Argument to activate
Leaky ReLU derivative
Arguments
- gradient :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Leaky ReLU derivative
Arguments
- gradient :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Returns a struct containing features, labels, as well as test_features and test_labels for the MNIST dataset
Computes the maxpooling of a Tensor
Arguments
- input :
Tensor-Tensorto pool - kernel : Tuple - Kernel height and width
- target :
Tensor-Tensortruth values - padding :
Tuple-Tuplewith height and width of the padding - stride :
Tuple-Tuplewith height and width of the stride
Computes the maxpooling gradient
Arguments
- shape :
Array- Shape of gradient output - max_indices :
Tensor- Pooled max indices - grad_output :
Tensor- Output from forward pass
Mean relative error for Tensor, mean of the element-wise |y_true - y|/max(|y_true|, |y|) Normally the relative error is defined as |y_true - y| / |y_true|, but here max is used to make it symmetric and to prevent dividing by zero, guaranteed to return zero in the case when both values are zero.
Mean squared error loss
Arguments
- input :
Tensor- Predicted values - target :
Tensor- Truth values
Computes gradients of mean squared error loss
Arguments
- gradient :
Tensor-Tensorgradient computed from MSE forwards - cache :
Tensor4D - CachedTensorfrom activation - target :
Tensor-Tensortruth values
Compute numerical gradient for any function w.r.t. to an input Tensor, useful for gradient checking, recommend using float64 types to assure numerical precision. The gradient is calculated as: (f(x + h) - f(x - h)) / (2*h) where h is a small number, typically 1e-5 f(x) will be called for each input elements with +h and -h pertubation. Iterate over all elements calculating each partial derivative
Compute numerical gradient for any function w.r.t. to an input value, useful for gradient checking, recommend using float64 types to assure numerical precision. The gradient is calculated as: (f(x + h) - f(x - h)) / (2*h) where h is a small number, typically 1e-5.
ReLU activation function
Arguments
- x :
Tensor- Argument to activate
ReLU activation function
Arguments
- x :
Tensor- Argument to activate
ReLU activation function
Arguments
- x :
Tensor- Argument to activate
ReLU activation function
Arguments
- x :
Tensor- Argument to activate
Derivative of the ReLU activation function
Arguments
- gradient :
Tensor-Tensorto derive - cached :
TensorCachedTensorfrom activation
Derivative of the ReLU activation function
Arguments
- gradient :
Tensor-Tensorto derive - cached :
TensorCachedTensorfrom activation
Sigmoid takes a real value as input and outputs another value between 0 and 1. It’s easy to work with and has all the nice properties of activation functions: it’s non-linear, continuously differentiable, monotonic, and has a fixed output range.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.sigmoid(a) # => [0.524979, 0.584191, 0.65701 ]Sigmoid takes a real value as input and outputs another value between 0 and 1. It’s easy to work with and has all the nice properties of activation functions: it’s non-linear, continuously differentiable, monotonic, and has a fixed output range.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.sigmoid(a) # => [0.524979, 0.584191, 0.65701 ]Sigmoid takes a real value as input and outputs another value between 0 and 1. It’s easy to work with and has all the nice properties of activation functions: it’s non-linear, continuously differentiable, monotonic, and has a fixed output range.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.sigmoid(a) # => [0.524979, 0.584191, 0.65701 ]Sigmoid takes a real value as input and outputs another value between 0 and 1. It’s easy to work with and has all the nice properties of activation functions: it’s non-linear, continuously differentiable, monotonic, and has a fixed output range.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.sigmoid(a) # => [0.524979, 0.584191, 0.65701 ]Sigmoid cross entropy loss
Arguments
- input :
Tensor- Predicted values - target :
Tensor- Truth values
Sigmoid cross entropy loss
Arguments
- input :
Tensor- Predicted values - target :
Tensor- Truth values
Computes gradients of sigmoid cross entropy loss
Arguments
- gradient :
Tensor-Tensorgradient computed from SCE forwards - cache :
Tensor4D - CachedTensorfrom activation - target :
Tensor-Tensortruth values
Computes gradients of sigmoid cross entropy loss
Arguments
- gradient :
Tensor-Tensorgradient computed from SCE forwards - cache :
Tensor4D - CachedTensorfrom activation - target :
Tensor-Tensortruth values
Derivative of the Sigmoid function
Arguments
- gradient :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.d_sigmoid(a) # => [0.249376, 0.242912, 0.225348]Derivative of the Sigmoid function
Arguments
- gradient :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Examples
a = [0.1, 0.34, 0.65].to_tensor
puts Num::NN.d_sigmoid(a) # => [0.249376, 0.242912, 0.225348]Computes softmax cross entropy loss
Arguments
- input :
Tensor- Predicted values - target :
Tensor- Truth values
Computes gradients of SmCE loss
Arguments
- gradient :
Tensor-Tensorgradient computed from SmCE forwards - cache :
Tensor4D - CachedTensorfrom activation - target :
Tensor-Tensortruth values
Tanh squashes a real-valued number to the range [-1, 1]. It’s non-linear. But unlike Sigmoid, its output is zero-centered. Therefore, in practice the tanh non-linearity is always preferred to the sigmoid nonlinearity.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.tanh(a) # => [0.099668, 0.327477, 0.57167 ]Tanh squashes a real-valued number to the range [-1, 1]. It’s non-linear. But unlike Sigmoid, its output is zero-centered. Therefore, in practice the tanh non-linearity is always preferred to the sigmoid nonlinearity.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.tanh(a) # => [0.099668, 0.327477, 0.57167 ]Tanh squashes a real-valued number to the range [-1, 1]. It’s non-linear. But unlike Sigmoid, its output is zero-centered. Therefore, in practice the tanh non-linearity is always preferred to the sigmoid nonlinearity.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.tanh(a) # => [0.099668, 0.327477, 0.57167 ]Tanh squashes a real-valued number to the range [-1, 1]. It’s non-linear. But unlike Sigmoid, its output is zero-centered. Therefore, in practice the tanh non-linearity is always preferred to the sigmoid nonlinearity.
Arguments
- x :
Tensor-Tensorto activate
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.tanh(a) # => [0.099668, 0.327477, 0.57167 ]Derivative of the Tanh function
Arguments
- x :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.d_tanh(a) # => [0.990066, 0.892759, 0.673193]Derivative of the Tanh function
Arguments
- x :
Tensor-Tensorto derive - cached :
Tensor- CachedTensorfrom activation
Examples
a = [0.1, 0.34, 0.65].to_tensor
Num::NN.d_tanh(a) # => [0.990066, 0.892759, 0.673193]Nested types
- Num::NN::AdamOptimizer(T)
- Num::NN::ConvolutionalLayer(T)
- Num::NN::Distribution
- Num::NN::DropoutLayer(T)
- Num::NN::EluLayer(T)
- Num::NN::FanMode
- Num::NN::FlattenLayer(T)
- Num::NN::InputLayer(T)
- Num::NN::Layer(T)
- Num::NN::LeakyReluLayer(T)
- Num::NN::LinearLayer(T)
- Num::NN::Loss(T)
- Num::NN::MSELoss(T)
- Num::NN::MaxPoolLayer(T)
- Num::NN::Network(T)
- Num::NN::NetworkInfo(T)
- Num::NN::Optimizer(T)
- Num::NN::ReluLayer(T)
- Num::NN::SGDMomentumOptimizer(T)
- Num::NN::SGDOptimizer(T)
- Num::NN::SigmoidCrossEntropyLoss(T)
- Num::NN::SigmoidLayer(T)
- Num::NN::SoftmaxCrossEntropyLoss(T)