Let’s consider again the motor trends car data with target variable gpm100 and 10 predictors.
It is possible that a subject-matter expert might have strongly-held ideas regarding which of the 10 predictors should be used to predict gpm100. In this case, you should directly estimate the expert-indicated model.
In the event that no strong theory holds, you are faced with considering the presence or absence of each of 10 predictors in the model, which means that there are 1,024 (including the empty model) competing models involving these predictors. How would you even begin to look at these competing models? It is possible that some of the predictors are redundant, while others are more fundamental. You could inspect the original correlations, or you could use methods such as Principal Components Analysis or Factor Analysis to...
