Probit and logit models

Previously, we discussed different types of problems that can be solved with regression models. In particular, the dependent variable is continuous, such as house prices, salaries, and so on. A natural question is if dependent variables are not continuous – in other words, if they are categorical – how would we adapt our regression equation to predict a categorical response variable? For instance, a human resources department in a company wants to conduct an attrition study to predict whether an employee will stay with the company or a car dealership wants to know if one car can be sold or not based on prices, car models, colors, and so on.

First, we will study binary classification. Here, the outcome (dependent variable) is a binary response such as yes/no or to do/not to do. Let’s look back at the simple linear regression model:

y = β 0 + β 1 x+ ϵ

Here, the predicted outcome is a line crossing data...