12.6 HOW MANY COMPONENTS SHOULD WE EXTRACT?
Recall that one of the motivations for PCA was to reduce the dimensionality. The question arises, “How do we determine how many components to extract?” For example, should we retain only the first two principal components, since they explain over half (52% Cumulative Var) of the total variability? Or should we retain all five components, since they explain 100% of the variability? Well, clearly, retaining all five components does not help us to reduce the dimensionality. As usual, the answer lies somewhere between these two extremes.
12.6.1 The Eigenvalue Criterion
In Figure 12.8, the eigenvalues are labeled as “SS loadings.” An eigenvalue of 1.0 would mean that the component would explain about “one predictor’s worth” of the total variability. The rationale for using the eigenvalue criterion is that each component should explain at least one predictor's worth of the variability, and therefore...