English
Menu

Machine Learning Basics

In a three-class classification, the values input to the softmax function (logits) are 3, 1 and 0. About how many times the second class's output (probability) is the first class's output? Which is closest?

  1. A2 times
  2. B3 times
  3. C7.4 times
Show answer and explanation

Answer: C. 7.4 times

The softmax function turns each value zᵢ into exp(zᵢ) ÷ Σexp(zⱼ), giving probabilities that sum to 1. The denominator is the same for every class, so the ratio of two classes' outputs is exp(3) ÷ exp(1) = e² ≈ 7.4. The ratio of the outputs thus depends only on the difference of the logits, and a difference of 2 makes the probability about 7.4 times larger. Adding the same number to all the logits does not change the outputs.

  • A. This is the difference of the logits. In the softmax function, the difference becomes a ratio through the exponential function.
  • B. This is the ratio of the logits, 3 ÷ 1, not the ratio of the softmax outputs.
  • C. Correct. exp(3) ÷ exp(1) = e² ≈ 7.4.
Practice this topic Back to topic

Related calculators

Confusion matrix calculatorCompute accuracy, precision, recall, F1, specificity, MCC and other metrics from TP, FN, FP and TNCNN output size calculatorStack convolution, pooling and fully connected layers and get each layer's output size, parameter count and multiply-accumulate count

Books on Machine Learning Basics Ad

As an Amazon Associate I earn from qualifying purchases.

More Machine Learning Basics questions

  1. A small neural network with inputs x₁ = 1 and x₂ = 2 has two hidden neurons h₁ = ReLU(x₁ + x₂ − 1) and h₂ = ReLU(x₁ − 2x₂ + 1), and output y = 2h₁ + 3h₂ + 1. What is the value of y?
  2. Let p be the probability that a model outputs for the correct class. Which of the following describes the cross-entropy loss −log p used to train classifiers?
  3. In gradient descent, the gradient of the loss ∂L/∂w at weight w = 2.0 is 0.5 and the learning rate is 0.1. What is the weight after one update?
  4. When the derivative dy/dx of y = (2x + 1)² at x = 1 is found by the chain rule with u = 2x + 1 and y = u², which value is correct?
  5. In a neural network with many stacked layers, which of the following is the reason that using the sigmoid function as the activation function makes the layers near the input hard to train?