Penalized logistic regression
As the name suggests, a penalized logistic regression model includes an additional penalty term in the loss function of the usual logistic regression model. Recall that a standard logistic regression model seeks to minimize the negative log-likelihood function (or equivalently, maximize the log-likelihood function), defined as follows:
Q(𝜷) = 1 _ N ∑ i=1 N − [ y i logp i + (1 − y i)log(1 − p i)]
Here, p i = 1 _ 1 + e −(β 0+β 1x 1 (i)+β 2x 2 (i)+…+β px p (i)) is the predicted probability for input x (i), y i is the corresponding target label, and 𝜷 = { β 0, β 1, … , β p} are model parameters to be estimated. Note that we now express the loss as a function of the coefficient vector as it...
