Machine Learning Basics
Which of the following correctly describes the ROC curve and AUC used to evaluate two-class classification models?
- AA curve drawn by varying the decision threshold and plotting the true positive rate against the false positive rate; the closer the area under it (AUC) is to 1, the better the model separates positives from negatives
- BA curve of the accuracy at a fixed threshold plotted against the number of training passes (epochs), where AUC is the maximum of that curve and a larger value means faster training
- CA model with an AUC of 0.5 correctly separates positives from negatives at least half the time, so it can be considered good enough for practical use
Show answer and explanation
Answer: A. A curve drawn by varying the decision threshold and plotting the true positive rate against the false positive rate; the closer the area under it (AUC) is to 1, the better the model separates positives from negatives
The ROC curve is drawn by moving the threshold above which the model's output score is judged positive, plotting the false positive rate FP ÷ (FP + TN) on the horizontal axis and the true positive rate TP ÷ (TP + FN) on the vertical axis. AUC is the area under the curve, and it equals the probability that a randomly chosen positive example scores higher than a randomly chosen negative example. A model that separates the classes perfectly has an AUC of 1, and one that judges at random has 0.5. AUC compares models' ability to discriminate before a threshold is chosen, but for data with very few positives the PR curve, which plots precision against recall, can show differences more clearly.
- A. Correct. It shows the relationship between the true positive rate and the false positive rate as the threshold changes, and the closer AUC is to 1, the better the model discriminates.
- B. A curve that shows the progress of training is a learning curve. The ROC curve shows the relationship between the true positive rate and the false positive rate as the threshold changes.
- C. An AUC of 0.5 is the same as judging at random and means the model has no ability to discriminate.