Model comparison and selection

We have explored various machine learning techniques and built several models to predict the credit ratings of customers, so now comes the question of which model we should select and how the models compare against each other. Our test data has 130 instances of customers with a bad credit rating (0) and 270 customers with a good credit rating (1).

If you remember, earlier we had talked about using domain knowledge and business requirements after doing modeling to interpret results and make decisions. Right now, our decision is to choose the best model to maximize profits and minimize losses for the German bank. Let us consider the following conditions:

• If we incorrectly predict a customer with bad credit rating as good, the bank will end up losing the whole credit amount lent to him since he will default on the payment and so loss is 100%, which can be denoted as -1 for our ease of calculation.

• If we correctly predict a customer with bad credit rating as bad,...