Machine Learning Basics
Which of the following describes principal component analysis (PCA)?
- ASelecting the few features that best predict the correct-answer labels and discarding all the other features to make the data smaller
- BDividing data into a predetermined number of groups so that similar items fall in the same group
- CFinding new, mutually orthogonal axes in order, starting from the direction in which the spread (variance) of the data is largest, and representing the data with few axes
Show answer and explanation
Answer: C. Finding new, mutually orthogonal axes in order, starting from the direction in which the spread (variance) of the data is largest, and representing the data with few axes
Principal component analysis is a representative dimensionality-reduction method that represents data with many variables using fewer variables while losing as little information as possible. It finds, in order, the direction of greatest variance in the data as the first principal component, the direction of greatest variance orthogonal to it as the second principal component, and so on, and represents the data with only the leading components. It is a kind of unsupervised learning that uses no correct-answer labels and is used for visualizing data and for preprocessing before training. If the variables have different units or scales, the result is pulled toward variables with large values, so the data is often standardized first.
- A. Choosing features with the labels is feature selection. PCA uses no labels and builds new axes that combine the original variables.
- B. This describes clustering (k-means and others).
- C. Correct. Mutually orthogonal axes (principal components) are found in order of decreasing variance.