Modeling and evaluation

Now we'll discuss various aspects of modeling and assessment. In both the KNN and SVM cases, we'll do feature selection using a technique known as Recursive Feature Elimination (RFE) in conjunction with cross-validation. As with all feature reduction and selection, this will help to prevent overfitting the model.

KNN modeling

As stated previously, we'll begin with feature selection. The caret package helps out in this matter. In RFE, a model is built using all features, and a feature importance value is assigned. Then the features are recursively pruned and an optimal number of features selected based on a performance metric such as accuracy. In short, it's a type of backward feature...