Implementing decision trees

Decision trees are the most widely used data mining machine learning algorithm in practice for classification and regression. They are easy to interpret, handle categorical features and extend to the multiclass classification. This decision tree model, which is a powerful, non-probabilistic technique, captures more complex nonlinear patterns and feature interactions. Their outcome is quite understandable. They are not hard to use since it's not required to tweak a lot of parameters.

This recipe shows how to run the decision tree on web content which evaluates a large set of URLs and classifies them as ephemeral (that is, short-lived and will cease being popular soon) or evergreen (that last for longer time). It is available in the Spark MLlib package. The code is written in Scala.

Getting ready

To step through this recipe, you will need a running Spark cluster in any one of the modes, that is, local, standalone, YARN, or Mesos. For installing Spark on a standalone...