Machine Learning Basics
Which of the following is correct when a model's error is considered in terms of bias and variance?
- AA model with large bias changes its predictions greatly depending on which training data is chosen, and tends to overfit
- BMaking a model more complex reduces both bias and variance, so it is best to use as complex a model as possible
- CMaking a model more complex tends to reduce bias but increase variance, so a complexity that balances the two is chosen
Show answer and explanation
Answer: C. Making a model more complex tends to reduce bias but increase variance, so a complexity that balances the two is chosen
For squared error, the expected error on new data can be decomposed into the sum of three terms: the squared bias, the variance, and the noise in the data itself. Bias is the error caused by the model being too simple to capture the true relationship, and large bias leads to underfitting. Variance is the error caused by predictions varying with the choice of training data, and large variance leads to overfitting. In general, making a model more complex lowers bias and raises variance, so a complexity that balances the two is chosen using a validation set or cross-validation.
- A. A model whose predictions change greatly with the training data and that tends to overfit is one with large variance.
- B. In general, making a model more complex lowers bias but raises variance.
- C. Correct. Bias and variance trade off against each other, so a complexity that balances them is chosen.