k-NN classifier
In Chapter 1, Introduction to Ensemble Techniques, we became familiar with a variety of classification models. Some readers might already be familiar with the k-NN model. The k-NN classifier is one of the most simple, intuitive, and non-assumptive models. The name of the model itself suggests how it might be working - nearest neighborhoods! And that's preceded by k! Thus, if we have N points in a study, we find the k-nearest points in neighborhood, and then make a note of the class of the k-neighbors. The majority class of the k-neighbors is then assigned to the unit. In case of regression, the average of the neighbors is assigned to the unit. The following is a visual depiction of k-NN:
Figure 4: Visual depiction of k-NN
The top left part of the visual depiction of k-NN shows the scatterplot of 27 observations, 16 of which are circles and the remaining 11 are squares. The circles are marked in orange 
while the squares are marked in blue 
. Suppose we choose to set up a classifier...
