The concept of Cost-Sensitive Learning

Cost-Sensitive Learning (CSL) is a technique where the cost function of a machine learning model is changed to account for the imbalance in data. The key insight behind CSL is that we want our model’s cost function to reflect the relative importance of the different classes.

Let’s try to understand cost functions in machine learning and various types of CSL.

Costs and cost functions

A cost function estimates the difference between the actual outcome and the predicted outcome from a model. For example, the cost function of the logistic regression model is given by the log loss function:

LogLoss = −  1 _ N * ∑ i=1  N  ( y i * log( ˆ y  i) + (1 − y i)* log(1 −  ˆ y  i))

Here, N is the total number of observations, y i is the true label (0 or 1), and  ˆ y  i is the...