English
Menu

Machine Learning Basics

A small neural network with inputs x₁ = 1 and x₂ = 2 has two hidden neurons h₁ = ReLU(x₁ + x₂ − 1) and h₂ = ReLU(x₁ − 2x₂ + 1), and output y = 2h₁ + 3h₂ + 1. What is the value of y?

  1. A−1
  2. B5
  3. C7
Show answer and explanation

Answer: B. 5

ReLU is an activation function that outputs its input unchanged when positive and 0 when the input is 0 or less. h₁ = ReLU(1 + 2 − 1) = ReLU(2) = 2 and h₂ = ReLU(1 − 4 + 1) = ReLU(−2) = 0, so y = 2 × 2 + 3 × 0 + 1 = 5. Computing layer by layer from the input to the output in this way is called the forward pass (forward propagation).

  • A. This is 2 × 2 + 3 × (−2) + 1 = −1, computed with h₂ = −2 without applying ReLU.
  • B. Correct. h₁ = 2 and h₂ = 0, so y = 2 × 2 + 3 × 0 + 1 = 5.
  • C. This is 2 × 3 + 1 = 7, with h₁ = 3 and h₂ = 0 obtained by forgetting the hidden-layer biases (−1 and +1).
Practice this topic Back to topic

Related calculators

Confusion matrix calculatorCompute accuracy, precision, recall, F1, specificity, MCC and other metrics from TP, FN, FP and TNCNN output size calculatorStack convolution, pooling and fully connected layers and get each layer's output size, parameter count and multiply-accumulate count

Books on Machine Learning Basics Ad

As an Amazon Associate I earn from qualifying purchases.

More Machine Learning Basics questions

  1. Let p be the probability that a model outputs for the correct class. Which of the following describes the cross-entropy loss −log p used to train classifiers?
  2. In gradient descent, the gradient of the loss ∂L/∂w at weight w = 2.0 is 0.5 and the learning rate is 0.1. What is the weight after one update?
  3. When the derivative dy/dx of y = (2x + 1)² at x = 1 is found by the chain rule with u = 2x + 1 and y = u², which value is correct?
  4. In a neural network with many stacked layers, which of the following is the reason that using the sigmoid function as the activation function makes the layers near the input hard to train?
  5. A model is trained on 10,000 training samples with a mini-batch size (batch size) of 50. How many weight updates are there in one epoch?