Classification Using K-Nearest Neighbors
Now that we are comfortable with creating multiclass classifiers using logistic regression and are getting reasonable performance with these models, we will turn our attention to another type of classifier: the K-nearest neighbors (KNN) classifier. KNN is a non-probabilistic, non-linear classifier. It does not predict the probability of a class. Also, as it does not learn any parameters, there is no linear combination of parameters and, thus, it is a non-linear model:
Figure 5.24: Visual representation of KNN
Figure 5.24 represents the workings of a KNN classifier. The two different symbols, X and O, represent data points belonging to two different classes. The solid circle at the center is the test point requiring classification, the inner dotted circle shows the classification process where k=3, while the outer dotted circle shows the classification process where k=5. What we mean here is that, if k=3, we only look...
