Comparing classification methods
In this chapter we have examined classification using logistic regression, support vector machines, and gradient boosted decision trees. In what scenarios should we prefer one algorithm over another?
For logistic regression, the data ideally will be linearly separable (the exponent in the formula for the logistic regression, after all, is essentially the same as the SVM equation for a separating hyperplane). If our goal is inference (producing a unit increase in response per 1-unit increase of input measurement, as we described in Chapter 1, From Data to Decisions – Getting Started with Analytic Applications) then the coefficients and log-odds values will be helpful. The stochastic gradient method can also be helpful in cases where we are unable to process all the data concurrently, while the second order methods we discussed may be easier to employ on un-normalized data. Finally, in the context of serializing model parameters and using these results to score...
