Logistic Regression
Logistic regression is the most favorable white-box model used for binary classification. White-box models are defined as models where we have visibility of the entire reasoning used for the prediction. For each prediction made, we can leverage the model's mathematical equation and decode the reasons for the prediction made. There are also a set of classification models that are completely black-box, that is, by no means can we understand the reasoning for the prediction leveraged by the model. In situations where we want to focus on only the end outcome, we should prefer black-box models, as they are more powerful.
Though the name ends with regression, logistic regression is a technique used to predict binary categorical outcomes. We would need a different approach to model for a categorical outcome. This can be done by transforming the outcome into a log of odds ratio or the probability of the event happening.
Let's distill this approach into simpler constructs. Assume...
