English
Menu

Machine Learning Basics

Which of the following correctly describes regularization and related methods for reducing overfitting?

  1. AL1 regularization adds the sum of the absolute values of the weights to the loss, and tends to make some weights exactly zero
  2. BL2 regularization adds the sum of the absolute values of the weights to the loss, and strongly narrows down features by setting unneeded weights to zero
  3. CDropout randomly disables neurons every time, in both training and inference, to give the output some variability
Show answer and explanation

Answer: A. L1 regularization adds the sum of the absolute values of the weights to the loss, and tends to make some weights exactly zero

Regularization adds a penalty depending on the size of the weights to the loss function, keeping the weights from growing too large and the model simple. L1 regularization adds the sum of the absolute values of the weights and tends to make some weights exactly zero, so it narrows down the features used (lasso regression). L2 regularization adds the sum of the squared weights and shrinks the weights overall, but they rarely become exactly zero (ridge regression, weight decay). Dropout randomly disables neurons during training only, so that the network does not rely too much on particular neurons; at inference all neurons are used.

  • A. Correct. The absolute-value penalty tends to make some weights exactly zero.
  • B. L2 regularization adds the sum of the squared weights; it shrinks the weights overall but rarely makes them exactly zero. Adding the sum of absolute values is L1 regularization.
  • C. Neurons are disabled only during training; all neurons are used at inference.
Practice this topic Back to topic

Related calculators

Confusion matrix calculatorCompute accuracy, precision, recall, F1, specificity, MCC and other metrics from TP, FN, FP and TNCNN output size calculatorStack convolution, pooling and fully connected layers and get each layer's output size, parameter count and multiply-accumulate count

Books on Machine Learning Basics Ad

As an Amazon Associate I earn from qualifying purchases.

More Machine Learning Basics questions

  1. Which of the following steps in building a machine learning model causes data leakage (information from the evaluation data slipping into training)?
  2. A two-class classification model was evaluated on 200 test samples, and its confusion matrix shows 45 true positives (TP), 5 false positives (FP), 45 false negatives (FN) and 105 true negatives (TN). Which pair of precision and recall is correct?
  3. A two-class classification model was evaluated on 200 test samples, giving TP = 20, FP = 5, FN = 30 and TN = 145. Which is closest to the F1 score?
  4. Which of the following correctly describes the ROC curve and AUC used to evaluate two-class classification models?
  5. For data of 1000 items of which only 10 are positive, what is the accuracy of a model that predicts every item as negative?